[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:41 dist-test-slave-2j7r krb5kdc[31505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:41 dist-test-slave-2j7r krb5kdc[31505](info): set up 2 sockets
Aug 08 17:01:41 dist-test-slave-2j7r krb5kdc[31505](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 17:01:42.526476 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:01:42 dist-test-slave-2j7r krb5kdc[31505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:01:42.636899 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:41589
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:41589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:42.715436 31524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:42.715621 31524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:42.715651 31524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:42.716735 31524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:42.716776 31524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:42.716794 31524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:42.716809 31524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:42.716825 31524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:42.718438 31524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:41589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:41589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:42.718693 31524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:42.718909 31524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:42.723526 31535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:42.723520 31537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:42.723503 31534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:42.723759 31524 server_base.cc:1034] running on GCE node
I20260808 17:01:42.723994 31524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:42.724246 31524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:42.725382 31524 hybrid_clock.cc:648] HybridClock initialized: now 1786208502725371 us; error 30 us; skew 500 ppm
Aug 08 17:01:42 dist-test-slave-2j7r krb5kdc[31505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208502, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:42.727653 31524 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:01:42.728060 31524 webserver.cc:460] Webserver started at http://127.30.194.62:40897/ using document root <none> and password file <none>
I20260808 17:01:42.728247 31524 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:42.728293 31524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:42.728382 31524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:42.729415 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "0272ee1812bf47b593160801a5462ba3"
format_stamp: "Formatted at 2026-08-08 17:01:42 on dist-test-slave-2j7r"
server_key: "a1d465af83fb1cf64764785d17f6073b"
server_key_iv: "3534eae362729c1c82d163428958a525"
server_key_version: "encryptionkey@0"
I20260808 17:01:42.729702 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "0272ee1812bf47b593160801a5462ba3"
format_stamp: "Formatted at 2026-08-08 17:01:42 on dist-test-slave-2j7r"
server_key: "a1d465af83fb1cf64764785d17f6073b"
server_key_iv: "3534eae362729c1c82d163428958a525"
server_key_version: "encryptionkey@0"
I20260808 17:01:42.730856 31524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:42.731539 31545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:42.731684 31524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:42.731770 31524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "0272ee1812bf47b593160801a5462ba3"
format_stamp: "Formatted at 2026-08-08 17:01:42 on dist-test-slave-2j7r"
server_key: "a1d465af83fb1cf64764785d17f6073b"
server_key_iv: "3534eae362729c1c82d163428958a525"
server_key_version: "encryptionkey@0"
I20260808 17:01:42.731853 31524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:42.747354 31524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:42.749476 31524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:42.749604 31524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:42.753386 31524 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:41589
I20260808 17:01:42.753424 31629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:41589 every 8 connection(s)
I20260808 17:01:42.753720 31524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:42.754571 31630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:42.756860 31630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3: Bootstrap starting.
I20260808 17:01:42.757454 31630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:42.757714 31630 log.cc:824] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:42.758396 31630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3: No bootstrap required, opened a new log
I20260808 17:01:42.759910 31630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0272ee1812bf47b593160801a5462ba3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41589 } }
I20260808 17:01:42.760037 31630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:42.760057 31630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0272ee1812bf47b593160801a5462ba3, State: Initialized, Role: FOLLOWER
I20260808 17:01:42.760159 31630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0272ee1812bf47b593160801a5462ba3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41589 } }
I20260808 17:01:42.760207 31630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:42.760233 31630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:42.760263 31630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:42.760730 31630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0272ee1812bf47b593160801a5462ba3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41589 } }
I20260808 17:01:42.760839 31630 leader_election.cc:302] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0272ee1812bf47b593160801a5462ba3; no voters:
I20260808 17:01:42.760994 31630 leader_election.cc:288] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:42.761029 31636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:42.761106 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 31524
I20260808 17:01:42.761160 31636 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [term 1 LEADER]: Becoming Leader. State: Replica: 0272ee1812bf47b593160801a5462ba3, State: Running, Role: LEADER
I20260808 17:01:42.761217 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:01:42.761245 31636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0272ee1812bf47b593160801a5462ba3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41589 } }
I20260808 17:01:42.761307 31630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:42.761545 31637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0272ee1812bf47b593160801a5462ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0272ee1812bf47b593160801a5462ba3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41589 } } }
I20260808 17:01:42.761616 31637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:42.761762 31638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0272ee1812bf47b593160801a5462ba3. Latest consensus state: current_term: 1 leader_uuid: "0272ee1812bf47b593160801a5462ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0272ee1812bf47b593160801a5462ba3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41589 } } }
I20260808 17:01:42.761811 31638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:42.761837 31640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:42.761986 31496 external_mini_cluster.cc:1428] Setting key 8bfe4f85a9d136dc6d4e52773ddc2d11
I20260808 17:01:42.762368 31640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:42.764307 31640 catalog_manager.cc:1347] Generated new cluster ID: cdaad95e358e41359602f534c0a9d56d
I20260808 17:01:42.764359 31640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:01:42 dist-test-slave-2j7r krb5kdc[31505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:42.776649 31640 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:42.777304 31640 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:42.779147 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:42.763176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.30.194.62:41589)
0808 17:01:42.763339 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:42.763347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:42.763377 (+ 30us) server_negotiation.cc:407] Connection header received
0808 17:01:42.765469 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:42.765476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:42.765530 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:42.765667 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:42.767518 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:42.769375 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:42.770712 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:42.771301 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:42.771409 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:42.775437 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:42.775441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:42.775441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:42.775452 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:42.777372 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:42.777826 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:42.777828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:42.777828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:42.777922 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:42.778177 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:42.778178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:42.778178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:42.778418 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:42.778541 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:42.778809 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 17:01:42.778937 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260808 17:01:42.783461 31640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0272ee1812bf47b593160801a5462ba3: Generated new TSK 0
I20260808 17:01:42.783607 31640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:01:42.807452 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:41589
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:42.888635 31669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:42.888813 31669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:42.888834 31669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:42.889967 31669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:42.890008 31669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:42.890034 31669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:01:42.891429 31669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:41589
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:42.891626 31669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:42.891832 31669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:42.896380 31679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:42.896415 31682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:42.896422 31676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:42.896557 31669 server_base.cc:1034] running on GCE node
I20260808 17:01:42.896795 31669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:42.897022 31669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:42.898183 31669 hybrid_clock.cc:648] HybridClock initialized: now 1786208502898177 us; error 43 us; skew 500 ppm
Aug 08 17:01:42 dist-test-slave-2j7r krb5kdc[31505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208502, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:42.900552 31669 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:01:42.900960 31669 webserver.cc:460] Webserver started at http://127.30.194.1:36803/ using document root <none> and password file <none>
I20260808 17:01:42.901208 31669 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:42.901268 31669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:42.901388 31669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:42.902328 31669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "64f9ee1a558d4ce5a4cb650d289fd07e"
format_stamp: "Formatted at 2026-08-08 17:01:42 on dist-test-slave-2j7r"
server_key: "c8e1b94ee52e7d6f10856d7237bed68b"
server_key_iv: "f76ecffcfe5497da21bb4da95f3461aa"
server_key_version: "encryptionkey@0"
I20260808 17:01:42.902639 31669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "64f9ee1a558d4ce5a4cb650d289fd07e"
format_stamp: "Formatted at 2026-08-08 17:01:42 on dist-test-slave-2j7r"
server_key: "c8e1b94ee52e7d6f10856d7237bed68b"
server_key_iv: "f76ecffcfe5497da21bb4da95f3461aa"
server_key_version: "encryptionkey@0"
I20260808 17:01:42.903816 31669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:01:42.904453 31694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:42.904593 31669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:01:42.904686 31669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "64f9ee1a558d4ce5a4cb650d289fd07e"
format_stamp: "Formatted at 2026-08-08 17:01:42 on dist-test-slave-2j7r"
server_key: "c8e1b94ee52e7d6f10856d7237bed68b"
server_key_iv: "f76ecffcfe5497da21bb4da95f3461aa"
server_key_version: "encryptionkey@0"
I20260808 17:01:42.904763 31669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:42.920728 31669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:42.922405 31669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:42.922521 31669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:42.922712 31669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:42.922973 31669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:42.923004 31669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:42.923022 31669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:42.923045 31669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:42.928510 31669 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:44107
I20260808 17:01:42.928555 31893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:44107 every 8 connection(s)
I20260808 17:01:42.928829 31669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:01:42.931528 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 31669
I20260808 17:01:42.931631 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:01:42.931792 31496 external_mini_cluster.cc:1428] Setting key e2cb9364cf0457453aaf47581d94fca1
Aug 08 17:01:42 dist-test-slave-2j7r krb5kdc[31505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208502, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:42.938454 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:42.929629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:47581 (local address 127.30.194.62:41589)
0808 17:01:42.929674 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:42.929676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:42.929875 (+ 199us) server_negotiation.cc:407] Connection header received
0808 17:01:42.930306 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:42.930307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:42.930318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:42.930376 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:42.931416 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:42.932158 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:42.933192 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:42.933455 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:42.933521 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:42.935784 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:42.935785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:42.935785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:42.935792 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:42.937230 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:42.937596 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:42.937597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:42.937598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:42.937661 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:42.937865 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:42.937866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:42.937866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:42.938000 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:42.938072 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:42.938274 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 17:01:42.938361 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:01:42.938524 31900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:42.929592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:41589 (local address 127.30.194.1:47581)
0808 17:01:42.929756 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:42.929795 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:42.930229 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:42.930433 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:42.930436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:42.931081 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:42.931301 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:42.931306 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:42.932263 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:42.932264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:42.933119 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:42.933123 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:42.933530 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:42.933531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:42.933627 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:42.934095 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:42.934105 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:42.935691 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:42.937308 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:42.937314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:42.937316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:42.937524 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:42.937722 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:42.937722 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:42.937722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:42.937812 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:42.938059 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:42.938061 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:42.938162 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 17:01:42.938234 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:01:42.938368 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260808 17:01:42.938977 31896 heartbeater.cc:347] Connected to a master server at 127.30.194.62:41589
I20260808 17:01:42.939059 31896 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:42.939224 31896 heartbeater.cc:511] Master 127.30.194.62:41589 requested a full tablet report, sending...
I20260808 17:01:42.939733 31573 ts_manager.cc:194] Registered new tserver with Master: 64f9ee1a558d4ce5a4cb650d289fd07e (127.30.194.1:44107)
I20260808 17:01:42.940330 31573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:47581
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:01:42.956034 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:41589
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:43.036080 31908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:43.036278 31908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:43.036311 31908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:43.037436 31908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:43.037484 31908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:43.037524 31908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:01:43.038944 31908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:41589
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:43.039161 31908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:43.039381 31908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:43.044095 31920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:43.044128 31924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:43.044106 31919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:43.044363 31908 server_base.cc:1034] running on GCE node
I20260808 17:01:43.044585 31908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:43.044775 31908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:43.045943 31908 hybrid_clock.cc:648] HybridClock initialized: now 1786208503045921 us; error 35 us; skew 500 ppm
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[31505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208503, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:43.048346 31908 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:01:43.048767 31908 webserver.cc:460] Webserver started at http://127.30.194.2:42581/ using document root <none> and password file <none>
I20260808 17:01:43.048980 31908 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:43.049031 31908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:43.049192 31908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:43.050159 31908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "7d7f5b620fa647da8433548dfc42f3ee"
format_stamp: "Formatted at 2026-08-08 17:01:43 on dist-test-slave-2j7r"
server_key: "9bd35b241b869e7c4ee994c0e92e30bf"
server_key_iv: "a5afd00bb074df64376fd3a303353b23"
server_key_version: "encryptionkey@0"
I20260808 17:01:43.050506 31908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "7d7f5b620fa647da8433548dfc42f3ee"
format_stamp: "Formatted at 2026-08-08 17:01:43 on dist-test-slave-2j7r"
server_key: "9bd35b241b869e7c4ee994c0e92e30bf"
server_key_iv: "a5afd00bb074df64376fd3a303353b23"
server_key_version: "encryptionkey@0"
I20260808 17:01:43.051810 31908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:43.052536 31938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:43.052711 31908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:43.052783 31908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "7d7f5b620fa647da8433548dfc42f3ee"
format_stamp: "Formatted at 2026-08-08 17:01:43 on dist-test-slave-2j7r"
server_key: "9bd35b241b869e7c4ee994c0e92e30bf"
server_key_iv: "a5afd00bb074df64376fd3a303353b23"
server_key_version: "encryptionkey@0"
I20260808 17:01:43.052848 31908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:43.089615 31908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:43.091380 31908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:43.091527 31908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:43.091761 31908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:43.092082 31908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:43.092118 31908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:43.092146 31908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:43.092163 31908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:43.098311 31908 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:44833
I20260808 17:01:43.098358 32124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:44833 every 8 connection(s)
I20260808 17:01:43.098645 31908 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:01:43.100297 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 31908
I20260808 17:01:43.100407 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:01:43.100603 31496 external_mini_cluster.cc:1428] Setting key b1f9710e31acb45664c3beeac3041a95
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[31505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208503, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:43.108423 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:43.099374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:45763 (local address 127.30.194.62:41589)
0808 17:01:43.099451 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:43.099453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:43.099615 (+ 162us) server_negotiation.cc:407] Connection header received
0808 17:01:43.100085 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:43.100086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:43.100096 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:43.100133 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:43.101340 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.101842 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.103187 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.103458 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.103523 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:43.105538 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:43.105540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:43.105540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:43.105549 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:43.107109 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:43.107510 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:43.107512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:43.107512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:43.107579 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:43.107796 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:43.107796 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:43.107796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:43.107927 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:43.107992 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:43.108243 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 17:01:43.108330 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 17:01:43.108620 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:43.099340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:41589 (local address 127.30.194.2:45763)
0808 17:01:43.099497 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:43.099534 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:43.099996 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:43.100234 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:43.100239 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:43.100929 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:43.101265 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:43.101272 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.101935 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.101936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:43.103077 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:43.103082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.103532 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.103533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:43.103631 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:43.104086 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:43.104094 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:43.105437 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:43.107227 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:43.107232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:43.107234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:43.107442 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:43.107649 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:43.107649 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:43.107649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:43.107745 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:43.107988 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:43.107991 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:43.108124 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 17:01:43.108209 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 17:01:43.108387 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260808 17:01:43.109054 32125 heartbeater.cc:347] Connected to a master server at 127.30.194.62:41589
I20260808 17:01:43.109163 32125 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:43.109323 32125 heartbeater.cc:511] Master 127.30.194.62:41589 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
I20260808 17:01:43.109732 31573 ts_manager.cc:194] Registered new tserver with Master: 7d7f5b620fa647da8433548dfc42f3ee (127.30.194.2:44833)
I20260808 17:01:43.110087 31573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:45763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:01:43.124936 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:41589
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:43.205828 32134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:43.206039 32134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:43.206065 32134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:43.207137 32134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:43.207176 32134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:43.207211 32134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:01:43.208696 32134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:41589
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:43.208876 32134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:43.209059 32134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:43.213861 32143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:43.213992 32134 server_base.cc:1034] running on GCE node
W20260808 17:01:43.213868 32152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:43.213883 32142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:43.214278 32134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:43.214501 32134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:43.215647 32134 hybrid_clock.cc:648] HybridClock initialized: now 1786208503215633 us; error 28 us; skew 500 ppm
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[31505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208503, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:43.218190 32134 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:01:43.218590 32134 webserver.cc:460] Webserver started at http://127.30.194.3:46419/ using document root <none> and password file <none>
I20260808 17:01:43.218802 32134 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:43.218845 32134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:43.218936 32134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:43.219863 32134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "3e931c5de20541ecb346f229d33758cd"
format_stamp: "Formatted at 2026-08-08 17:01:43 on dist-test-slave-2j7r"
server_key: "948dd42a7293ee8ddb5de56027ba0f61"
server_key_iv: "4089aec755ef8755928e83a390b93d7f"
server_key_version: "encryptionkey@0"
I20260808 17:01:43.220145 32134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "3e931c5de20541ecb346f229d33758cd"
format_stamp: "Formatted at 2026-08-08 17:01:43 on dist-test-slave-2j7r"
server_key: "948dd42a7293ee8ddb5de56027ba0f61"
server_key_iv: "4089aec755ef8755928e83a390b93d7f"
server_key_version: "encryptionkey@0"
I20260808 17:01:43.221393 32134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:43.222018 32165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:43.222177 32134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:43.222270 32134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "3e931c5de20541ecb346f229d33758cd"
format_stamp: "Formatted at 2026-08-08 17:01:43 on dist-test-slave-2j7r"
server_key: "948dd42a7293ee8ddb5de56027ba0f61"
server_key_iv: "4089aec755ef8755928e83a390b93d7f"
server_key_version: "encryptionkey@0"
I20260808 17:01:43.222324 32134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:43.237972 32134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:43.239710 32134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:43.239857 32134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:43.240064 32134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:43.240330 32134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:43.240362 32134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:43.240387 32134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:43.240413 32134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:43.246137 32134 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:41283
I20260808 17:01:43.246196 32351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:41283 every 8 connection(s)
I20260808 17:01:43.246448 32134 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:01:43.249163 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 32134
I20260808 17:01:43.249253 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:01:43.249406 31496 external_mini_cluster.cc:1428] Setting key bea7fe0058b9c4a7f177cf4a0d90254b
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[31505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208503, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:43.254599 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:43.247079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:59969 (local address 127.30.194.62:41589)
0808 17:01:43.247134 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:43.247135 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:43.247332 (+ 197us) server_negotiation.cc:407] Connection header received
0808 17:01:43.247797 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:43.247798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:43.247810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:43.247858 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:43.248735 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.249262 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.250218 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.250466 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.250533 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:43.252346 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:43.252347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:43.252347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:43.252354 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:43.253593 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:43.253919 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:43.253920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:43.253920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:43.253963 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:43.254128 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:43.254129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:43.254129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:43.254253 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:43.254303 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:43.254480 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 17:01:43.254538 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:01:43.254739 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:43.247062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:41589 (local address 127.30.194.3:59969)
0808 17:01:43.247218 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:43.247256 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:43.247701 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:43.247942 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:43.247947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:43.248474 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:43.248647 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:43.248651 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.249362 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.249363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:43.250157 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:43.250159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.250563 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.250564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:43.250662 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:43.251009 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:43.251017 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:43.252271 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:43.253666 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:43.253671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:43.253673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:43.253867 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:43.254006 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:43.254006 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:43.254007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:43.254088 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:43.254298 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:43.254301 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:43.254397 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 17:01:43.254465 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 17:01:43.254568 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260808 17:01:43.255124 32352 heartbeater.cc:347] Connected to a master server at 127.30.194.62:41589
I20260808 17:01:43.255199 32352 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:43.255393 32352 heartbeater.cc:511] Master 127.30.194.62:41589 requested a full tablet report, sending...
I20260808 17:01:43.255738 31573 ts_manager.cc:194] Registered new tserver with Master: 3e931c5de20541ecb346f229d33758cd (127.30.194.3:41283)
I20260808 17:01:43.256065 31573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:59969
I20260808 17:01:43.261240 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[31505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208503, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[31505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208503, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:01:43.284305 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:43.277349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.30.194.1:44107)
0808 17:01:43.277553 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:43.277557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:43.277568 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:01:43.277592 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:43.277594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:43.277609 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:43.277684 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:43.278164 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.278895 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.279602 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.279916 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.279978 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:43.281841 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:43.281847 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:43.281848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:43.281854 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:43.283148 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:43.283481 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:43.283483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:43.283483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:43.283541 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:43.283747 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:43.283748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:43.283749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:43.283873 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:43.284033 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:43.284097 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 17:01:43.284174 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
W20260808 17:01:43.284631 31738 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46484
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[31505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[31505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:01:43.304687 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:43.298816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.30.194.1:44107)
0808 17:01:43.298874 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:43.298876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:43.298936 (+ 60us) server_negotiation.cc:407] Connection header received
0808 17:01:43.298976 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:43.298977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:43.298986 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:43.299035 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:43.299545 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.300001 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.300655 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:43.300850 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:43.300898 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:43.302735 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:43.302736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:43.302736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:43.302743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:43.303811 (+ 1068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:43.304084 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:43.304085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:43.304085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:43.304134 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:43.304357 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:43.304358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:43.304358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:43.304435 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:43.304510 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:43.304579 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 17:01:43.304633 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:01:43.305193 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 31669
I20260808 17:01:43.314479 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 31908
I20260808 17:01:43.329880 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 32134
I20260808 17:01:43.337459 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 31524
2026-08-08T17:01:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1998 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[32375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[32375](info): set up 2 sockets
Aug 08 17:01:43 dist-test-slave-2j7r krb5kdc[32375](info): commencing operation
krb5kdc: starting...
W20260808 17:01:44.473805 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:01:44 dist-test-slave-2j7r krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:01:44.581379 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:42539
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:42539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:44.659507 32400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:44.659691 32400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:44.659726 32400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:44.660841 32400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:44.660878 32400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:44.660897 32400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:44.660912 32400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:44.660928 32400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:44.662438 32400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:42539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:42539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:44.662793 32400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:44.663022 32400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:44.667739 32409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:44.667718 32407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:44.667714 32406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:44.667778 32400 server_base.cc:1034] running on GCE node
I20260808 17:01:44.668119 32400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:44.668392 32400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:44.669543 32400 hybrid_clock.cc:648] HybridClock initialized: now 1786208504669523 us; error 29 us; skew 500 ppm
Aug 08 17:01:44 dist-test-slave-2j7r krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208504, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:44.676471 32400 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:01:44.676894 32400 webserver.cc:460] Webserver started at http://127.30.194.62:43525/ using document root <none> and password file <none>
I20260808 17:01:44.677112 32400 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:44.677167 32400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:44.677277 32400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:44.678242 32400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "4e265795f5a34069ac3b02f8fb902c89"
format_stamp: "Formatted at 2026-08-08 17:01:44 on dist-test-slave-2j7r"
server_key: "c0b34c3418219e83efd23a1eab42ad85"
server_key_iv: "bdad11a335a8fe1a61eb3b2e88f890c1"
server_key_version: "encryptionkey@0"
I20260808 17:01:44.678546 32400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "4e265795f5a34069ac3b02f8fb902c89"
format_stamp: "Formatted at 2026-08-08 17:01:44 on dist-test-slave-2j7r"
server_key: "c0b34c3418219e83efd23a1eab42ad85"
server_key_iv: "bdad11a335a8fe1a61eb3b2e88f890c1"
server_key_version: "encryptionkey@0"
I20260808 17:01:44.679806 32400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:44.680512 32418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:44.680675 32400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:44.680768 32400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "4e265795f5a34069ac3b02f8fb902c89"
format_stamp: "Formatted at 2026-08-08 17:01:44 on dist-test-slave-2j7r"
server_key: "c0b34c3418219e83efd23a1eab42ad85"
server_key_iv: "bdad11a335a8fe1a61eb3b2e88f890c1"
server_key_version: "encryptionkey@0"
I20260808 17:01:44.680819 32400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:44.694712 32400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:44.696496 32400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:44.696622 32400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:44.700268 32400 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:42539
I20260808 17:01:44.700300 32492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:42539 every 8 connection(s)
I20260808 17:01:44.700593 32400 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:44.701398 32494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:44.703429 32494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89: Bootstrap starting.
I20260808 17:01:44.703951 32494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:44.704195 32494 log.cc:824] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:44.704823 32494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89: No bootstrap required, opened a new log
I20260808 17:01:44.705327 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 32400
I20260808 17:01:44.705415 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:01:44.705567 31496 external_mini_cluster.cc:1428] Setting key ea99661e320bb4a9c5f81034816887af
I20260808 17:01:44.706736 32494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e265795f5a34069ac3b02f8fb902c89" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42539 } }
I20260808 17:01:44.706873 32494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:44.706910 32494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e265795f5a34069ac3b02f8fb902c89, State: Initialized, Role: FOLLOWER
I20260808 17:01:44.707033 32494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e265795f5a34069ac3b02f8fb902c89" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42539 } }
I20260808 17:01:44.707113 32494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:44.707149 32494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:44.707195 32494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:44.708086 32494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e265795f5a34069ac3b02f8fb902c89" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42539 } }
I20260808 17:01:44.708182 32494 leader_election.cc:302] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e265795f5a34069ac3b02f8fb902c89; no voters:
I20260808 17:01:44.708333 32494 leader_election.cc:288] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:44.708364 32506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:44.708465 32506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [term 1 LEADER]: Becoming Leader. State: Replica: 4e265795f5a34069ac3b02f8fb902c89, State: Running, Role: LEADER
I20260808 17:01:44.708567 32494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:44.708544 32506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e265795f5a34069ac3b02f8fb902c89" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42539 } }
I20260808 17:01:44.708879 32506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e265795f5a34069ac3b02f8fb902c89. Latest consensus state: current_term: 1 leader_uuid: "4e265795f5a34069ac3b02f8fb902c89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e265795f5a34069ac3b02f8fb902c89" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42539 } } }
I20260808 17:01:44.708951 32506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:44.709121 32510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e265795f5a34069ac3b02f8fb902c89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e265795f5a34069ac3b02f8fb902c89" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42539 } } }
I20260808 17:01:44.709187 32510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:44.709311 32514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:44.709776 32514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:01:44 dist-test-slave-2j7r krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:44.714111 32514 catalog_manager.cc:1347] Generated new cluster ID: 701fa61c8d0e4fc19f04fd627858c6ff
I20260808 17:01:44.714166 32514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:44.715535 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:44.706030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36044 (local address 127.30.194.62:42539)
0808 17:01:44.706185 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:44.706192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:44.706223 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:01:44.706694 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:44.706705 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:44.706746 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:44.706848 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:44.707569 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:44.708551 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:44.709619 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:44.710197 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:44.710305 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:44.712423 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:44.712427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:44.712427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:44.712437 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:44.714083 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:44.714434 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:44.714437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:44.714437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:44.714544 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:44.714726 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:44.714726 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:44.714727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:44.714944 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:44.715084 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:44.715176 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 17:01:44.715390 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260808 17:01:44.737912 32514 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:44.738386 32514 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:44.740763 32514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e265795f5a34069ac3b02f8fb902c89: Generated new TSK 0
I20260808 17:01:44.740908 32514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:01:44.769548 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42539
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:44.849915 32542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:44.850126 32542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:44.850152 32542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:44.851281 32542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:44.851326 32542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:44.851357 32542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:01:44.852866 32542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42539
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:44.853178 32542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:44.853413 32542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:44.858275 32553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:44.858361 32550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:44.858280 32551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:44.858475 32542 server_base.cc:1034] running on GCE node
I20260808 17:01:44.858695 32542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:44.858958 32542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:44.860113 32542 hybrid_clock.cc:648] HybridClock initialized: now 1786208504860092 us; error 34 us; skew 500 ppm
Aug 08 17:01:44 dist-test-slave-2j7r krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208504, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:44.862730 32542 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:01:44.863168 32542 webserver.cc:460] Webserver started at http://127.30.194.1:43385/ using document root <none> and password file <none>
I20260808 17:01:44.863410 32542 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:44.863466 32542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:44.863601 32542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:44.864754 32542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "77ca9b5b77f444778dc658c750a79772"
format_stamp: "Formatted at 2026-08-08 17:01:44 on dist-test-slave-2j7r"
server_key: "fb3f780e729543a6c814ce83f67a9e13"
server_key_iv: "73b7d2a1e0744a11de3d603c46193451"
server_key_version: "encryptionkey@0"
I20260808 17:01:44.865187 32542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "77ca9b5b77f444778dc658c750a79772"
format_stamp: "Formatted at 2026-08-08 17:01:44 on dist-test-slave-2j7r"
server_key: "fb3f780e729543a6c814ce83f67a9e13"
server_key_iv: "73b7d2a1e0744a11de3d603c46193451"
server_key_version: "encryptionkey@0"
I20260808 17:01:44.867022 32542 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 17:01:44.868016 32562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:44.868202 32542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:44.868301 32542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "77ca9b5b77f444778dc658c750a79772"
format_stamp: "Formatted at 2026-08-08 17:01:44 on dist-test-slave-2j7r"
server_key: "fb3f780e729543a6c814ce83f67a9e13"
server_key_iv: "73b7d2a1e0744a11de3d603c46193451"
server_key_version: "encryptionkey@0"
I20260808 17:01:44.868358 32542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:44.883915 32542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:44.885880 32542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:44.886000 32542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:44.886168 32542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:44.886479 32542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:44.886514 32542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:44.886543 32542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:44.886567 32542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:44.891681 32542 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:40681
I20260808 17:01:44.891741 32746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:40681 every 8 connection(s)
I20260808 17:01:44.891991 32542 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:01:44.893746 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 32542
I20260808 17:01:44.893860 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:01:44.894037 31496 external_mini_cluster.cc:1428] Setting key d115522458bf698ce23ee4a9dc50b439
Aug 08 17:01:44 dist-test-slave-2j7r krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208504, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:44.901975 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:44.892673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:43651 (local address 127.30.194.62:42539)
0808 17:01:44.892743 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:44.892746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:44.892886 (+ 140us) server_negotiation.cc:407] Connection header received
0808 17:01:44.894004 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:44.894005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:44.894019 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:44.894066 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:44.894808 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:44.895304 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:44.896618 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:44.896918 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:44.896995 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:44.899292 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:44.899294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:44.899294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:44.899304 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:44.900612 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:44.901054 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:44.901055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:44.901056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:44.901134 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:44.901399 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:44.901400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:44.901400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:44.901501 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:44.901553 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:44.901843 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 17:01:44.901911 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:01:44.902195 32749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:44.892627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:42539 (local address 127.30.194.1:43651)
0808 17:01:44.892777 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:44.892811 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:44.893894 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:44.894149 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:44.894153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:44.894477 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:44.894705 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:44.894712 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:44.895395 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:44.895396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:44.896525 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:44.896528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:44.896996 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:44.896997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:44.897123 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:44.897597 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:44.897606 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:44.899191 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:44.900695 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:44.900701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:44.900704 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:44.900984 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:44.901194 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:44.901195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:44.901195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:44.901338 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:44.901558 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:44.901564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:44.901713 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 17:01:44.901828 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 17:01:44.901980 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
I20260808 17:01:44.902740 32747 heartbeater.cc:347] Connected to a master server at 127.30.194.62:42539
I20260808 17:01:44.902832 32747 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:44.902998 32747 heartbeater.cc:511] Master 127.30.194.62:42539 requested a full tablet report, sending...
I20260808 17:01:44.903517 32441 ts_manager.cc:194] Registered new tserver with Master: 77ca9b5b77f444778dc658c750a79772 (127.30.194.1:40681)
I20260808 17:01:44.904119 32441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:43651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:01:44.916953 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42539
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:44.994175 32756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:44.994355 32756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:44.994377 32756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:44.995486 32756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:44.995524 32756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:44.995553 32756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:01:44.997006 32756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42539
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:44.997231 32756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:44.997424 32756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:45.001951 300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:45.001989 301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:45.001966 306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:45.002236 32756 server_base.cc:1034] running on GCE node
I20260808 17:01:45.002414 32756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:45.002640 32756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:45.003787 32756 hybrid_clock.cc:648] HybridClock initialized: now 1786208505003767 us; error 33 us; skew 500 ppm
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208505, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:45.006168 32756 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:01:45.006608 32756 webserver.cc:460] Webserver started at http://127.30.194.2:34079/ using document root <none> and password file <none>
I20260808 17:01:45.006809 32756 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:45.006855 32756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:45.006974 32756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:45.007963 32756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "4e61bc1395814371b2c1bab9638221c8"
format_stamp: "Formatted at 2026-08-08 17:01:45 on dist-test-slave-2j7r"
server_key: "a10b093a3b433de3ecd1ccdab38cfc96"
server_key_iv: "c381b08d74f5b9eaf23004dfc46f2001"
server_key_version: "encryptionkey@0"
I20260808 17:01:45.008284 32756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "4e61bc1395814371b2c1bab9638221c8"
format_stamp: "Formatted at 2026-08-08 17:01:45 on dist-test-slave-2j7r"
server_key: "a10b093a3b433de3ecd1ccdab38cfc96"
server_key_iv: "c381b08d74f5b9eaf23004dfc46f2001"
server_key_version: "encryptionkey@0"
I20260808 17:01:45.009601 32756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:45.010259 316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:45.010411 32756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:45.010494 32756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "4e61bc1395814371b2c1bab9638221c8"
format_stamp: "Formatted at 2026-08-08 17:01:45 on dist-test-slave-2j7r"
server_key: "a10b093a3b433de3ecd1ccdab38cfc96"
server_key_iv: "c381b08d74f5b9eaf23004dfc46f2001"
server_key_version: "encryptionkey@0"
I20260808 17:01:45.010558 32756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:45.026882 32756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:45.028409 32756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:45.028529 32756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:45.028720 32756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:45.028980 32756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:45.029012 32756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:45.029038 32756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:45.029062 32756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:45.034729 32756 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:43147
I20260808 17:01:45.034776 515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:43147 every 8 connection(s)
I20260808 17:01:45.035053 32756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208505, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:45.040987 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 32756
I20260808 17:01:45.041070 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:01:45.041251 31496 external_mini_cluster.cc:1428] Setting key 8b212310116917c9c6fbe6f099a6d6bc
I20260808 17:01:45.043421 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.035720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:34619 (local address 127.30.194.62:42539)
0808 17:01:45.035774 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.035777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.035939 (+ 162us) server_negotiation.cc:407] Connection header received
0808 17:01:45.036806 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.036807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.036816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.036851 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:45.037392 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.037875 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.038773 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.038962 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.039011 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.040782 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:45.040784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:45.040784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:45.040792 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:45.042201 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.042613 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.042613 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.042614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.042668 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.042897 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.042897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.042897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.042986 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:45.043029 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.043300 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.043360 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:01:45.043630 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.035679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:42539 (local address 127.30.194.2:34619)
0808 17:01:45.035827 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:45.035864 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:45.036708 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:45.036908 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:45.036911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:45.037151 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:45.037333 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.037337 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.037931 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.037932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.038708 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.038710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.039018 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.039019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.039107 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.039489 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:45.039496 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:45.040700 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:45.042275 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:45.042282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:45.042284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:45.042545 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:45.042716 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:45.042717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:45.042717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:45.042846 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:45.043032 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:45.043036 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:45.043175 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 17:01:45.043281 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 17:01:45.043423 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260808 17:01:45.044049 522 heartbeater.cc:347] Connected to a master server at 127.30.194.62:42539
I20260808 17:01:45.044128 522 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:45.044284 522 heartbeater.cc:511] Master 127.30.194.62:42539 requested a full tablet report, sending...
I20260808 17:01:45.044624 32441 ts_manager.cc:194] Registered new tserver with Master: 4e61bc1395814371b2c1bab9638221c8 (127.30.194.2:43147)
I20260808 17:01:45.045056 32441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:34619
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:01:45.064842 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42539
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:45.147055 533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:45.147303 533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:45.147337 533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:45.148484 533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:45.148530 533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:45.148556 533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:01:45.150151 533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42539
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:45.150373 533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:45.150589 533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:45.155287 542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:45.155392 533 server_base.cc:1034] running on GCE node
W20260808 17:01:45.155309 543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:45.155309 545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:45.155750 533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:45.155988 533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:45.157159 533 hybrid_clock.cc:648] HybridClock initialized: now 1786208505157136 us; error 33 us; skew 500 ppm
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208505, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:45.159528 533 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:01:45.159924 533 webserver.cc:460] Webserver started at http://127.30.194.3:42327/ using document root <none> and password file <none>
I20260808 17:01:45.160113 533 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:45.160161 533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:45.160310 533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:45.161320 533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "1db6698430af4c378338676e63a10127"
format_stamp: "Formatted at 2026-08-08 17:01:45 on dist-test-slave-2j7r"
server_key: "fa5105092f8f893256b718752383b58d"
server_key_iv: "16547eebf1f3cba02afcc0a72cff9b61"
server_key_version: "encryptionkey@0"
I20260808 17:01:45.161638 533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "1db6698430af4c378338676e63a10127"
format_stamp: "Formatted at 2026-08-08 17:01:45 on dist-test-slave-2j7r"
server_key: "fa5105092f8f893256b718752383b58d"
server_key_iv: "16547eebf1f3cba02afcc0a72cff9b61"
server_key_version: "encryptionkey@0"
I20260808 17:01:45.162849 533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:45.163545 554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:45.163688 533 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:45.163774 533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "1db6698430af4c378338676e63a10127"
format_stamp: "Formatted at 2026-08-08 17:01:45 on dist-test-slave-2j7r"
server_key: "fa5105092f8f893256b718752383b58d"
server_key_iv: "16547eebf1f3cba02afcc0a72cff9b61"
server_key_version: "encryptionkey@0"
I20260808 17:01:45.163831 533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:45.183120 533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:45.184942 533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:45.185066 533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:45.185299 533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:45.185576 533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:45.185608 533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:45.185634 533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:45.185655 533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:45.191061 533 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:37519
I20260808 17:01:45.191125 749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:37519 every 8 connection(s)
I20260808 17:01:45.191390 533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208505, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:45.199249 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 533
I20260808 17:01:45.199338 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:01:45.199481 31496 external_mini_cluster.cc:1428] Setting key d07b2f2305a5a3187c9d325f09a99fa7
I20260808 17:01:45.200248 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.192053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:36135 (local address 127.30.194.62:42539)
0808 17:01:45.192095 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.192098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.192322 (+ 224us) server_negotiation.cc:407] Connection header received
0808 17:01:45.193262 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.193264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.193274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.193328 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:45.193879 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.194360 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.195328 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.195517 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.195576 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.197585 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:45.197586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:45.197586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:45.197592 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:45.198813 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.199177 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.199178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.199179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.199242 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.199521 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.199522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.199523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.199656 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:45.199716 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.200038 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.200125 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:01:45.200342 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.192022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:42539 (local address 127.30.194.3:36135)
0808 17:01:45.192180 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:45.192248 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:45.193162 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:45.193380 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:45.193383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:45.193624 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:45.193807 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.193811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.194439 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.194440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.195230 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.195233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.195583 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.195584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.195682 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.196117 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:45.196125 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:45.197499 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:45.198874 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:45.198879 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:45.198881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:45.199102 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:45.199335 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:45.199336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:45.199337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:45.199457 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:45.199791 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:45.199797 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:45.199906 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 17:01:45.199994 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 17:01:45.200138 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260808 17:01:45.200728 750 heartbeater.cc:347] Connected to a master server at 127.30.194.62:42539
I20260808 17:01:45.200817 750 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:45.200976 750 heartbeater.cc:511] Master 127.30.194.62:42539 requested a full tablet report, sending...
I20260808 17:01:45.201355 32441 ts_manager.cc:194] Registered new tserver with Master: 1db6698430af4c378338676e63a10127 (127.30.194.3:37519)
I20260808 17:01:45.201686 32441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:36135
I20260808 17:01:45.211051 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:01:45.218202 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.212564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36048 (local address 127.30.194.62:42539)
0808 17:01:45.212621 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.212623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.212708 (+ 85us) server_negotiation.cc:407] Connection header received
0808 17:01:45.212763 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.212764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.212773 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.212819 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:45.213460 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.213925 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.214614 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.214819 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.214872 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.215925 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:45.215927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:45.215927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:45.215935 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:45.217195 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.217492 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.217493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.217493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.217549 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.217705 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.217706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.217706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.217850 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:45.217938 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.218046 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.218118 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:01:45.220148 32441 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:01:45.221724 32441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:45.227613 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.223736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:43147 (local address 127.0.0.1:35468)
0808 17:01:45.224007 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:45.224047 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:45.224109 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:45.224594 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:45.224595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:45.224597 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:45.224748 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.224750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.225895 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.225896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.226669 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.226671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.227343 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.227344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.227422 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.227434 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 17:01:45.227494 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:01:45.227544 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":51,"threads_started":1}
I20260808 17:01:45.227623 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.223847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.30.194.2:43147)
0808 17:01:45.224447 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.224452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.224461 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:45.224501 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.224502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.224518 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.224591 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:45.224829 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.225808 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.226752 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.227266 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.227329 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.227386 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.227516 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.227554 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":74,"threads_started":1}
I20260808 17:01:45.227881 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.223890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.30.194.3:37519)
0808 17:01:45.224052 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.224057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.224191 (+ 134us) server_negotiation.cc:407] Connection header received
0808 17:01:45.224239 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.224241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.224253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.224315 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:45.224485 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.225351 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.226783 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.227465 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.227510 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.227560 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.227775 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.227812 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260808 17:01:45.228104 764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.223770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:37519 (local address 127.0.0.1:45322)
0808 17:01:45.224011 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:45.224050 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:45.224112 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:45.224336 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:45.224339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:45.224341 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:45.224402 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.224404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.225439 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.225440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.226671 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.226674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.227614 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.227615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.227669 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.227672 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:45.228005 (+ 333us) client_negotiation.cc:240] Negotiation successful
0808 17:01:45.228038 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":6,"thread_start_us":61,"threads_started":1}
I20260808 17:01:45.228276 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.224045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.30.194.1:40681)
0808 17:01:45.224258 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.224262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.224270 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:45.224300 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.224301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.224311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.224375 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:45.224498 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.225533 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.226805 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.227465 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.227528 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.227572 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.228147 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.228189 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":101,"threads_started":1}
I20260808 17:01:45.228103 763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.223914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:40681 (local address 127.0.0.1:58362)
0808 17:01:45.224091 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:45.224099 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:45.224141 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:45.224385 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:45.224386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:45.224389 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:45.224440 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.224442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.225628 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.225629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.226720 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.226722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.227605 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.227606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.227935 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.227940 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:01:45.227989 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 17:01:45.228027 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":2,"thread_start_us":37,"threads_started":1}
I20260808 17:01:45.228319 410 tablet_service.cc:1467] Processing CreateTablet for tablet f997dbcea8f94c268f9fd8f945d8e7b0 (DEFAULT_TABLE table=test-table [id=a88d38d3f28e473482d401ef4f38927c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:45.228688 410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f997dbcea8f94c268f9fd8f945d8e7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:45.228729 634 tablet_service.cc:1467] Processing CreateTablet for tablet f997dbcea8f94c268f9fd8f945d8e7b0 (DEFAULT_TABLE table=test-table [id=a88d38d3f28e473482d401ef4f38927c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:45.229050 634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f997dbcea8f94c268f9fd8f945d8e7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:45.229194 32644 tablet_service.cc:1467] Processing CreateTablet for tablet f997dbcea8f94c268f9fd8f945d8e7b0 (DEFAULT_TABLE table=test-table [id=a88d38d3f28e473482d401ef4f38927c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:45.229504 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f997dbcea8f94c268f9fd8f945d8e7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:45.231310 772 tablet_bootstrap.cc:492] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8: Bootstrap starting.
I20260808 17:01:45.231941 772 tablet_bootstrap.cc:654] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:45.232461 774 tablet_bootstrap.cc:492] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772: Bootstrap starting.
I20260808 17:01:45.232545 773 tablet_bootstrap.cc:492] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127: Bootstrap starting.
I20260808 17:01:45.232585 772 log.cc:824] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:45.233142 774 tablet_bootstrap.cc:654] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:45.233142 773 tablet_bootstrap.cc:654] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:45.233354 772 tablet_bootstrap.cc:492] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8: No bootstrap required, opened a new log
I20260808 17:01:45.233413 772 ts_tablet_manager.cc:1397] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:01:45.233683 773 log.cc:824] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:45.233683 774 log.cc:824] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:45.234460 774 tablet_bootstrap.cc:492] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772: No bootstrap required, opened a new log
I20260808 17:01:45.234534 774 ts_tablet_manager.cc:1397] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:01:45.234658 773 tablet_bootstrap.cc:492] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127: No bootstrap required, opened a new log
I20260808 17:01:45.234720 773 ts_tablet_manager.cc:1397] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:01:45.235080 772 raft_consensus.cc:348] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e61bc1395814371b2c1bab9638221c8" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 43147 } } peers { permanent_uuid: "77ca9b5b77f444778dc658c750a79772" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40681 } } peers { permanent_uuid: "1db6698430af4c378338676e63a10127" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37519 } }
I20260808 17:01:45.235214 772 raft_consensus.cc:374] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:45.235287 772 raft_consensus.cc:729] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e61bc1395814371b2c1bab9638221c8, State: Initialized, Role: FOLLOWER
I20260808 17:01:45.235414 772 consensus_queue.cc:260] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e61bc1395814371b2c1bab9638221c8" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 43147 } } peers { permanent_uuid: "77ca9b5b77f444778dc658c750a79772" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40681 } } peers { permanent_uuid: "1db6698430af4c378338676e63a10127" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37519 } }
I20260808 17:01:45.235675 772 ts_tablet_manager.cc:1428] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:01:45.235782 522 heartbeater.cc:503] Master 127.30.194.62:42539 was elected leader, sending a full tablet report...
I20260808 17:01:45.236439 774 raft_consensus.cc:348] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e61bc1395814371b2c1bab9638221c8" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 43147 } } peers { permanent_uuid: "77ca9b5b77f444778dc658c750a79772" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40681 } } peers { permanent_uuid: "1db6698430af4c378338676e63a10127" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37519 } }
I20260808 17:01:45.236609 774 raft_consensus.cc:374] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:45.236519 773 raft_consensus.cc:348] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e61bc1395814371b2c1bab9638221c8" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 43147 } } peers { permanent_uuid: "77ca9b5b77f444778dc658c750a79772" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40681 } } peers { permanent_uuid: "1db6698430af4c378338676e63a10127" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37519 } }
I20260808 17:01:45.236654 773 raft_consensus.cc:374] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:45.236652 774 raft_consensus.cc:729] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ca9b5b77f444778dc658c750a79772, State: Initialized, Role: FOLLOWER
I20260808 17:01:45.236691 773 raft_consensus.cc:729] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1db6698430af4c378338676e63a10127, State: Initialized, Role: FOLLOWER
I20260808 17:01:45.236761 774 consensus_queue.cc:260] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e61bc1395814371b2c1bab9638221c8" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 43147 } } peers { permanent_uuid: "77ca9b5b77f444778dc658c750a79772" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40681 } } peers { permanent_uuid: "1db6698430af4c378338676e63a10127" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37519 } }
I20260808 17:01:45.236794 773 consensus_queue.cc:260] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e61bc1395814371b2c1bab9638221c8" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 43147 } } peers { permanent_uuid: "77ca9b5b77f444778dc658c750a79772" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40681 } } peers { permanent_uuid: "1db6698430af4c378338676e63a10127" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37519 } }
I20260808 17:01:45.237031 774 ts_tablet_manager.cc:1428] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 17:01:45.237041 750 heartbeater.cc:503] Master 127.30.194.62:42539 was elected leader, sending a full tablet report...
I20260808 17:01:45.237040 32747 heartbeater.cc:503] Master 127.30.194.62:42539 was elected leader, sending a full tablet report...
I20260808 17:01:45.237031 773 ts_tablet_manager.cc:1428] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260808 17:01:45.285559 524 tablet.cc:2369] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:01:45.392670 32748 tablet.cc:2369] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:01:45.441953 751 tablet.cc:2369] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:01:45.477841 785 raft_consensus.cc:482] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:45.477953 785 raft_consensus.cc:504] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e61bc1395814371b2c1bab9638221c8" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 43147 } } peers { permanent_uuid: "77ca9b5b77f444778dc658c750a79772" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40681 } } peers { permanent_uuid: "1db6698430af4c378338676e63a10127" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37519 } }
I20260808 17:01:45.478264 785 leader_election.cc:288] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e61bc1395814371b2c1bab9638221c8 (127.30.194.2:43147), 77ca9b5b77f444778dc658c750a79772 (127.30.194.1:40681)
I20260808 17:01:45.480906 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.478538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:54993 (local address 127.30.194.2:43147)
0808 17:01:45.478591 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.478593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.478603 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:01:45.478673 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.478674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.478683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.478736 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:45.478871 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.479329 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.480245 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.480607 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.480669 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.480678 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.480806 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.480842 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:01:45.480906 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.478476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:43147 (local address 127.30.194.3:54993)
0808 17:01:45.478530 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:45.478546 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:45.478601 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:45.478749 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:45.478750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:45.478752 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:45.478819 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.478820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.479399 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.479400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.480167 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.480169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.480676 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.480676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.480749 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.480752 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:45.480796 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 17:01:45.480834 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":24}
I20260808 17:01:45.481057 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.478581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:40681 (local address 127.30.194.3:58551)
0808 17:01:45.478743 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:45.478751 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:45.478804 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:45.478922 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:45.478923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:45.478924 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:45.478989 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.478991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.479555 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.479556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.480326 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:45.480328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.480820 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.480821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:45.480893 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.480896 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:45.480941 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:01:45.480976 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260808 17:01:45.481050 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.478649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:58551 (local address 127.30.194.1:40681)
0808 17:01:45.478701 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.478703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.478808 (+ 105us) server_negotiation.cc:407] Connection header received
0808 17:01:45.478852 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.478853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.478866 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.478911 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:45.479045 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.479484 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.480411 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.480749 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.480807 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.480815 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.480956 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.480987 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:01:45.481305 449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f997dbcea8f94c268f9fd8f945d8e7b0" candidate_uuid: "1db6698430af4c378338676e63a10127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e61bc1395814371b2c1bab9638221c8" is_pre_election: true
I20260808 17:01:45.481457 449 raft_consensus.cc:2454] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1db6698430af4c378338676e63a10127 in term 0.
I20260808 17:01:45.481490 32680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f997dbcea8f94c268f9fd8f945d8e7b0" candidate_uuid: "1db6698430af4c378338676e63a10127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77ca9b5b77f444778dc658c750a79772" is_pre_election: true
I20260808 17:01:45.481617 32680 raft_consensus.cc:2454] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1db6698430af4c378338676e63a10127 in term 0.
I20260808 17:01:45.481706 557 leader_election.cc:302] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1db6698430af4c378338676e63a10127, 4e61bc1395814371b2c1bab9638221c8; no voters:
I20260808 17:01:45.481827 785 raft_consensus.cc:2781] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:45.481881 785 raft_consensus.cc:482] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:45.481909 785 raft_consensus.cc:3037] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:45.482539 785 raft_consensus.cc:504] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e61bc1395814371b2c1bab9638221c8" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 43147 } } peers { permanent_uuid: "77ca9b5b77f444778dc658c750a79772" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40681 } } peers { permanent_uuid: "1db6698430af4c378338676e63a10127" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37519 } }
I20260808 17:01:45.482640 785 leader_election.cc:288] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [CANDIDATE]: Term 1 election: Requested vote from peers 4e61bc1395814371b2c1bab9638221c8 (127.30.194.2:43147), 77ca9b5b77f444778dc658c750a79772 (127.30.194.1:40681)
I20260808 17:01:45.482820 449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f997dbcea8f94c268f9fd8f945d8e7b0" candidate_uuid: "1db6698430af4c378338676e63a10127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e61bc1395814371b2c1bab9638221c8"
I20260808 17:01:45.482843 32680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f997dbcea8f94c268f9fd8f945d8e7b0" candidate_uuid: "1db6698430af4c378338676e63a10127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77ca9b5b77f444778dc658c750a79772"
I20260808 17:01:45.482903 32680 raft_consensus.cc:3037] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:45.482903 449 raft_consensus.cc:3037] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:45.483768 449 raft_consensus.cc:2454] T f997dbcea8f94c268f9fd8f945d8e7b0 P 4e61bc1395814371b2c1bab9638221c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1db6698430af4c378338676e63a10127 in term 1.
I20260808 17:01:45.483768 32680 raft_consensus.cc:2454] T f997dbcea8f94c268f9fd8f945d8e7b0 P 77ca9b5b77f444778dc658c750a79772 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1db6698430af4c378338676e63a10127 in term 1.
I20260808 17:01:45.483914 557 leader_election.cc:302] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1db6698430af4c378338676e63a10127, 77ca9b5b77f444778dc658c750a79772; no voters:
I20260808 17:01:45.484004 785 raft_consensus.cc:2781] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:45.484148 785 raft_consensus.cc:686] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [term 1 LEADER]: Becoming Leader. State: Replica: 1db6698430af4c378338676e63a10127, State: Running, Role: LEADER
I20260808 17:01:45.484273 785 consensus_queue.cc:237] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e61bc1395814371b2c1bab9638221c8" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 43147 } } peers { permanent_uuid: "77ca9b5b77f444778dc658c750a79772" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40681 } } peers { permanent_uuid: "1db6698430af4c378338676e63a10127" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37519 } }
I20260808 17:01:45.485055 32441 catalog_manager.cc:5495] T f997dbcea8f94c268f9fd8f945d8e7b0 P 1db6698430af4c378338676e63a10127 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1db6698430af4c378338676e63a10127 (127.30.194.3). New cstate: current_term: 1 leader_uuid: "1db6698430af4c378338676e63a10127" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e61bc1395814371b2c1bab9638221c8" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 43147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77ca9b5b77f444778dc658c750a79772" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1db6698430af4c378338676e63a10127" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37519 } health_report { overall_health: HEALTHY } } }
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:01:45.510605 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.503978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58364 (local address 127.30.194.1:40681)
0808 17:01:45.504023 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.504025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.504140 (+ 115us) server_negotiation.cc:407] Connection header received
0808 17:01:45.504192 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.504192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.504202 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.504246 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:45.504755 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.505244 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.505885 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.506086 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.506138 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.508171 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:45.508181 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:45.508182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:45.508189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:45.509582 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.509886 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.509889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.509890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.509955 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.510120 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.510121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.510122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.510254 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:45.510383 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.510444 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.510519 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208505, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208505, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:01:45.532645 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.526511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58376 (local address 127.30.194.1:40681)
0808 17:01:45.526593 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.526594 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.526632 (+ 38us) server_negotiation.cc:407] Connection header received
0808 17:01:45.526669 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.526670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.526678 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.526735 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:45.527179 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.527627 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.528265 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.528441 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.528492 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.530411 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:45.530413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:45.530413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:45.530419 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:45.531715 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.532007 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.532008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.532009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.532064 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.532275 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.532275 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.532275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.532351 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:45.532415 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.532534 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.532590 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:01:45.552795 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:45.547053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.30.194.1:40681)
0808 17:01:45.547102 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:45.547104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:45.547153 (+ 49us) server_negotiation.cc:407] Connection header received
0808 17:01:45.547207 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:45.547208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:45.547215 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:45.547252 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:45.547691 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.548116 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.548747 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:45.548906 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:45.548945 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:45.550794 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:45.550795 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:45.550795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:45.550801 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:45.551895 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.552171 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.552172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.552172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.552264 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:45.552436 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:45.552437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:45.552437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:45.552527 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:45.552597 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:45.552663 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 17:01:45.552720 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 17:01:45.553259 32612 tablet_service.cc:2908] Rejecting scan request for tablet f997dbcea8f94c268f9fd8f945d8e7b0: Uninitialized: clean time has not yet been initialized
I20260808 17:01:45.553957 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 32542
I20260808 17:01:45.562799 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 32756
I20260808 17:01:45.568411 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 533
I20260808 17:01:45.575166 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 32400
2026-08-08T17:01:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2242 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[809](info): set up 2 sockets
Aug 08 17:01:45 dist-test-slave-2j7r krb5kdc[809](info): commencing operation
krb5kdc: starting...
W20260808 17:01:46.724026 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:01:46 dist-test-slave-2j7r krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:01:46.831766 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:44619
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:36479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:44619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:46.912864 832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:46.913045 832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:46.913064 832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:46.914162 832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:46.914201 832 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 17:01:46.914214 832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:46.914227 832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:46.914237 832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:46.914247 832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:46.915649 832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:44619
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:44619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:46.915834 832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:46.915994 832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:46.920346 841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:46.920357 843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:46.920406 846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:46.920572 832 server_base.cc:1034] running on GCE node
I20260808 17:01:46.920816 832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:46.921077 832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:46.922269 832 hybrid_clock.cc:648] HybridClock initialized: now 1786208506922255 us; error 33 us; skew 500 ppm
Aug 08 17:01:46 dist-test-slave-2j7r krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208506, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:46.924551 832 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:01:46.925053 832 webserver.cc:460] Webserver started at http://127.30.194.62:44007/ using document root <none> and password file <none>
I20260808 17:01:46.925295 832 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:46.925350 832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:46.925468 832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:46.926400 832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "f73ca68920c849c99e7a19f2163242ac"
format_stamp: "Formatted at 2026-08-08 17:01:46 on dist-test-slave-2j7r"
server_key: "8e7b3a89d39cfcde4dff32bf0e30c61c"
server_key_iv: "5ba006cd27c54c43747fa4f33bf5e066"
server_key_version: "encryptionkey@0"
I20260808 17:01:46.926674 832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "f73ca68920c849c99e7a19f2163242ac"
format_stamp: "Formatted at 2026-08-08 17:01:46 on dist-test-slave-2j7r"
server_key: "8e7b3a89d39cfcde4dff32bf0e30c61c"
server_key_iv: "5ba006cd27c54c43747fa4f33bf5e066"
server_key_version: "encryptionkey@0"
I20260808 17:01:46.927834 832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:01:46.928673 860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:46.928874 832 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:46.928977 832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "f73ca68920c849c99e7a19f2163242ac"
format_stamp: "Formatted at 2026-08-08 17:01:46 on dist-test-slave-2j7r"
server_key: "8e7b3a89d39cfcde4dff32bf0e30c61c"
server_key_iv: "5ba006cd27c54c43747fa4f33bf5e066"
server_key_version: "encryptionkey@0"
I20260808 17:01:46.929039 832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:46.950326 832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:46.951953 832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:46.952065 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:46.956823 832 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:44619
I20260808 17:01:46.956887 962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:44619 every 8 connection(s)
I20260808 17:01:46.957209 832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:46.957844 963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:46.959959 963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac: Bootstrap starting.
I20260808 17:01:46.960501 963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:46.960762 963 log.cc:824] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:46.961385 963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac: No bootstrap required, opened a new log
I20260808 17:01:46.963450 963 raft_consensus.cc:348] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ca68920c849c99e7a19f2163242ac" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 44619 } }
I20260808 17:01:46.963578 963 raft_consensus.cc:374] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:46.963610 963 raft_consensus.cc:729] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f73ca68920c849c99e7a19f2163242ac, State: Initialized, Role: FOLLOWER
I20260808 17:01:46.963737 963 consensus_queue.cc:260] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ca68920c849c99e7a19f2163242ac" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 44619 } }
I20260808 17:01:46.963798 963 raft_consensus.cc:388] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:46.963828 963 raft_consensus.cc:482] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:46.963874 963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:46.964344 963 raft_consensus.cc:504] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ca68920c849c99e7a19f2163242ac" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 44619 } }
I20260808 17:01:46.964433 963 leader_election.cc:302] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f73ca68920c849c99e7a19f2163242ac; no voters:
I20260808 17:01:46.964579 963 leader_election.cc:288] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:46.964623 968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:46.964769 968 raft_consensus.cc:686] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [term 1 LEADER]: Becoming Leader. State: Replica: f73ca68920c849c99e7a19f2163242ac, State: Running, Role: LEADER
I20260808 17:01:46.964820 963 sys_catalog.cc:564] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:46.964851 968 consensus_queue.cc:237] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ca68920c849c99e7a19f2163242ac" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 44619 } }
I20260808 17:01:46.965260 972 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f73ca68920c849c99e7a19f2163242ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ca68920c849c99e7a19f2163242ac" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 44619 } } }
I20260808 17:01:46.965283 969 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader f73ca68920c849c99e7a19f2163242ac. Latest consensus state: current_term: 1 leader_uuid: "f73ca68920c849c99e7a19f2163242ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ca68920c849c99e7a19f2163242ac" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 44619 } } }
I20260808 17:01:46.965479 969 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:46.965637 972 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:46.965739 981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:46.965956 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 832
I20260808 17:01:46.966044 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:01:46.966193 31496 external_mini_cluster.cc:1428] Setting key a45110a3f9b6d6f467d51895241aec36
I20260808 17:01:46.966331 981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:46.967782 981 catalog_manager.cc:1347] Generated new cluster ID: a430473c65b445abb8d48411e6a71b33
I20260808 17:01:46.967831 981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:01:46 dist-test-slave-2j7r krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:46.974270 981 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:46.974810 981 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:46.977497 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:46.966769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56534 (local address 127.30.194.62:44619)
0808 17:01:46.966997 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:46.967006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:46.967038 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:01:46.967145 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:46.967151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:46.967202 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:46.967363 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:46.968185 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:46.969610 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:46.970663 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:46.971272 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:46.971366 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:46.974017 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:46.974021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:46.974021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:46.974033 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:46.975938 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:46.976312 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:46.976315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:46.976316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:46.976429 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:46.976600 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:46.976600 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:46.976601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:46.976871 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:46.977032 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:46.977120 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 17:01:46.977280 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":69,"threads_started":1}
I20260808 17:01:46.988011 981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac: Generated new TSK 0
I20260808 17:01:46.988148 981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:01:47.015862 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:44619
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:36479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:47.094173 1004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:47.094364 1004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:47.094393 1004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:47.095490 1004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:47.095528 1004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:47.095556 1004 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:01:47.095578 1004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:01:47.096979 1004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.194.62:44619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:47.097218 1004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:47.097422 1004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:47.101998 1014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:47.102085 1004 server_base.cc:1034] running on GCE node
W20260808 17:01:47.101999 1019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:47.102023 1022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:47.102476 1004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:47.102691 1004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:47.103832 1004 hybrid_clock.cc:648] HybridClock initialized: now 1786208507103819 us; error 29 us; skew 500 ppm
Aug 08 17:01:47 dist-test-slave-2j7r krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208507, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:47.106715 1004 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:01:47.107224 1004 webserver.cc:460] Webserver started at http://127.30.194.1:40835/ using document root <none> and password file <none>
I20260808 17:01:47.107489 1004 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:47.107553 1004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:47.107709 1004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:47.108733 1004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "a124eec79334414ab930fdfb832aa7a1"
format_stamp: "Formatted at 2026-08-08 17:01:47 on dist-test-slave-2j7r"
server_key: "7e680bd3393a088c57cbc75b56a8da77"
server_key_iv: "befc261f28db3bc3e70196235d935901"
server_key_version: "encryptionkey@0"
I20260808 17:01:47.109050 1004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "a124eec79334414ab930fdfb832aa7a1"
format_stamp: "Formatted at 2026-08-08 17:01:47 on dist-test-slave-2j7r"
server_key: "7e680bd3393a088c57cbc75b56a8da77"
server_key_iv: "befc261f28db3bc3e70196235d935901"
server_key_version: "encryptionkey@0"
I20260808 17:01:47.110270 1004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:47.110904 1036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:47.111049 1004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:01:47.111129 1004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "a124eec79334414ab930fdfb832aa7a1"
format_stamp: "Formatted at 2026-08-08 17:01:47 on dist-test-slave-2j7r"
server_key: "7e680bd3393a088c57cbc75b56a8da77"
server_key_iv: "befc261f28db3bc3e70196235d935901"
server_key_version: "encryptionkey@0"
I20260808 17:01:47.111192 1004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:47.126902 1004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:47.128613 1004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:47.128746 1004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:47.129284 1004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:47.129323 1004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:47.129357 1004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:47.129385 1004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:01:47 dist-test-slave-2j7r krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208507, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:47.135818 1004 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:37785
I20260808 17:01:47.135890 1223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:37785 every 8 connection(s)
I20260808 17:01:47.136191 1004 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:01:47.138387 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.129795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:35263 (local address 127.30.194.62:44619)
0808 17:01:47.129864 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.129866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.130008 (+ 142us) server_negotiation.cc:407] Connection header received
0808 17:01:47.130534 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.130536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.130549 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.130622 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:47.131416 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.132182 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.133585 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.133801 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.133861 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.135920 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:47.135922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:47.135922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:47.135930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:47.137345 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.137705 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.137706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.137706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.137754 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.137928 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.137928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.137929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.138022 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:47.138066 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.138259 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.138323 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:01:47.138515 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.129730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:44619 (local address 127.30.194.1:35263)
0808 17:01:47.129880 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.129919 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.130423 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.130682 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.130687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.131062 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:47.131333 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.131339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.132284 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.132285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.133500 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.133503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.133887 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.133889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.134016 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.134448 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:47.134457 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:47.135812 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:47.137428 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.137435 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.137436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.137646 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.137800 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.137800 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.137800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.137887 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.138061 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:47.138064 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:47.138164 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.138236 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.138346 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260808 17:01:47.138967 1224 heartbeater.cc:347] Connected to a master server at 127.30.194.62:44619
I20260808 17:01:47.139048 1224 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:47.139216 1224 heartbeater.cc:511] Master 127.30.194.62:44619 requested a full tablet report, sending...
I20260808 17:01:47.139730 886 ts_manager.cc:194] Registered new tserver with Master: a124eec79334414ab930fdfb832aa7a1 (127.30.194.1:37785)
I20260808 17:01:47.140363 886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:35263
I20260808 17:01:47.140507 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 1004
I20260808 17:01:47.140615 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:01:47.140779 31496 external_mini_cluster.cc:1428] Setting key 544221f9131022a67de1ed717c82f05d
I20260808 17:01:47.145709 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.139911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:44619 (local address 127.30.194.1:35487)
0808 17:01:47.140027 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.140036 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.140078 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.140625 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.140626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.140833 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:47.140892 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.140894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.141791 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.141791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.142429 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.142432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.142751 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.142751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.142816 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.143132 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:47.143137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:47.143462 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:47.144884 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.144885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.144885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.145032 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.145232 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.145233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.145233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.145284 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.145495 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:47.145496 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:47.145529 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.145571 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.145639 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":61,"threads_started":1}
I20260808 17:01:47.145740 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.139981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:35487 (local address 127.30.194.62:44619)
0808 17:01:47.140534 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.140536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.140544 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:47.140560 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.140561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.140569 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.140616 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:47.140985 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.141675 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.142506 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.142683 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.142735 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.143544 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:47.143546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:47.143546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:47.143553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:47.144811 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.145112 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.145114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.145114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.145171 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.145333 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.145334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.145334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.145444 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:47.145505 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.145595 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.145665 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537}
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:01:47.165489 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:44619
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:36479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:47.244767 1247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:47.244959 1247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:47.244982 1247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:47.246114 1247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:47.246158 1247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:47.246176 1247 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:01:47.246194 1247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:01:47.247593 1247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.194.62:44619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:47.247812 1247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:47.248013 1247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:47.252679 1265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:47.252684 1260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:47.252669 1257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:47.252930 1247 server_base.cc:1034] running on GCE node
I20260808 17:01:47.253146 1247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:47.253345 1247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:47.254477 1247 hybrid_clock.cc:648] HybridClock initialized: now 1786208507254463 us; error 29 us; skew 500 ppm
Aug 08 17:01:47 dist-test-slave-2j7r krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208507, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:47.256696 1247 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:01:47.257117 1247 webserver.cc:460] Webserver started at http://127.30.194.2:38915/ using document root <none> and password file <none>
I20260808 17:01:47.257320 1247 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:47.257359 1247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:47.257449 1247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:47.258369 1247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "ea1ce95654c84b6689ff45b263f8b787"
format_stamp: "Formatted at 2026-08-08 17:01:47 on dist-test-slave-2j7r"
server_key: "93cb898cc5452d85c6379023d79d4f1a"
server_key_iv: "e7b2fd8d99e7306eabe8794ffabeffc4"
server_key_version: "encryptionkey@0"
I20260808 17:01:47.258651 1247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "ea1ce95654c84b6689ff45b263f8b787"
format_stamp: "Formatted at 2026-08-08 17:01:47 on dist-test-slave-2j7r"
server_key: "93cb898cc5452d85c6379023d79d4f1a"
server_key_iv: "e7b2fd8d99e7306eabe8794ffabeffc4"
server_key_version: "encryptionkey@0"
I20260808 17:01:47.259770 1247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:47.260378 1274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:47.260517 1247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:47.260594 1247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "ea1ce95654c84b6689ff45b263f8b787"
format_stamp: "Formatted at 2026-08-08 17:01:47 on dist-test-slave-2j7r"
server_key: "93cb898cc5452d85c6379023d79d4f1a"
server_key_iv: "e7b2fd8d99e7306eabe8794ffabeffc4"
server_key_version: "encryptionkey@0"
I20260808 17:01:47.260656 1247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:47.270188 1247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:47.271795 1247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:47.271893 1247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:47.272341 1247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:47.272380 1247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:47.272408 1247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:47.272429 1247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:01:47 dist-test-slave-2j7r krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208507, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:47.279189 1247 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:36615
I20260808 17:01:47.279342 1480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:36615 every 8 connection(s)
I20260808 17:01:47.279551 1247 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:01:47.279769 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 1247
I20260808 17:01:47.280110 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:01:47.280277 31496 external_mini_cluster.cc:1428] Setting key b9e1a3a6ef6f07afec1dba09fdb76530
I20260808 17:01:47.281996 1291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.272633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:44619 (local address 127.30.194.2:54273)
0808 17:01:47.272785 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.272822 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.273319 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.273550 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.273553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.273772 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:47.273964 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.273968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.274815 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.274817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.275666 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.275670 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.276119 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.276120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.276257 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.276717 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:47.276726 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:47.278842 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:47.280618 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.280624 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.280626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.280826 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.281002 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.281003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.281003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.281148 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.281438 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:47.281444 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:47.281581 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.281665 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.281805 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260808 17:01:47.282402 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.272905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:54273 (local address 127.30.194.62:44619)
0808 17:01:47.272949 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.272951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.272960 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:47.273422 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.273424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.273437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.273486 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:47.274049 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.274701 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.275772 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.276031 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.276092 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.278948 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:47.278949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:47.278950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:47.278959 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:47.280543 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.280890 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.280891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.280892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.280950 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.281232 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.281233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.281234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.281359 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:47.281413 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.282249 (+ 836us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.282324 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:01:47.282721 1483 heartbeater.cc:347] Connected to a master server at 127.30.194.62:44619
I20260808 17:01:47.282807 1483 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:47.282975 1483 heartbeater.cc:511] Master 127.30.194.62:44619 requested a full tablet report, sending...
I20260808 17:01:47.283282 886 ts_manager.cc:194] Registered new tserver with Master: ea1ce95654c84b6689ff45b263f8b787 (127.30.194.2:36615)
I20260808 17:01:47.283667 886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:54273
I20260808 17:01:47.289835 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.284751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:60109 (local address 127.30.194.62:44619)
0808 17:01:47.284794 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.284796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.284850 (+ 54us) server_negotiation.cc:407] Connection header received
0808 17:01:47.284892 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.284892 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.284901 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.284959 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:47.285294 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.285716 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.286640 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.286819 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.286866 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.287691 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:47.287692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:47.287692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:47.287698 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:47.288823 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.289194 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.289194 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.289195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.289241 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.289414 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.289414 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.289414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.289532 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:47.289578 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.289729 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.289782 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:01:47.289892 1492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.284672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:44619 (local address 127.30.194.2:60109)
0808 17:01:47.284798 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.284808 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.284857 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.284972 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.284973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.285159 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:47.285215 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.285217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.285808 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.285809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.286562 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.286566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.286882 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.286882 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.286955 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.287280 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:47.287285 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:47.287617 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:47.288896 (+ 1279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.288897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.288897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.289123 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.289297 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.289298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.289298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.289362 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.289586 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:47.289587 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:47.289632 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.289706 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.289802 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:01:47.305467 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:44619
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:36479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:47.384243 1496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:47.384430 1496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:47.384461 1496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:47.385622 1496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:47.385663 1496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:47.385682 1496 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:01:47.385699 1496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:01:47.387086 1496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.194.62:44619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:47.387287 1496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:47.387475 1496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:47.392033 1507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:47.392033 1509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:47.392020 1503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:47.392304 1496 server_base.cc:1034] running on GCE node
I20260808 17:01:47.392485 1496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:47.392671 1496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:47.393796 1496 hybrid_clock.cc:648] HybridClock initialized: now 1786208507393786 us; error 28 us; skew 500 ppm
Aug 08 17:01:47 dist-test-slave-2j7r krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208507, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:47.396112 1496 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:01:47.396510 1496 webserver.cc:460] Webserver started at http://127.30.194.3:46559/ using document root <none> and password file <none>
I20260808 17:01:47.396674 1496 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:47.396711 1496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:47.396804 1496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:47.397745 1496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0"
format_stamp: "Formatted at 2026-08-08 17:01:47 on dist-test-slave-2j7r"
server_key: "c8c00960d3f137fbab9241bb9723df0e"
server_key_iv: "065b70e08017d30da70b85692d5f95d3"
server_key_version: "encryptionkey@0"
I20260808 17:01:47.398027 1496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0"
format_stamp: "Formatted at 2026-08-08 17:01:47 on dist-test-slave-2j7r"
server_key: "c8c00960d3f137fbab9241bb9723df0e"
server_key_iv: "065b70e08017d30da70b85692d5f95d3"
server_key_version: "encryptionkey@0"
I20260808 17:01:47.399238 1496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:01:47.399830 1520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:47.399987 1496 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:01:47.400068 1496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0"
format_stamp: "Formatted at 2026-08-08 17:01:47 on dist-test-slave-2j7r"
server_key: "c8c00960d3f137fbab9241bb9723df0e"
server_key_iv: "065b70e08017d30da70b85692d5f95d3"
server_key_version: "encryptionkey@0"
I20260808 17:01:47.400120 1496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:47.418318 1496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:47.420218 1496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:47.420322 1496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:47.420765 1496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:47.420804 1496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:47.420823 1496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:47.420835 1496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:01:47 dist-test-slave-2j7r krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208507, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:47.427631 1496 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:39383
I20260808 17:01:47.427695 1713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:39383 every 8 connection(s)
I20260808 17:01:47.428000 1496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:01:47.428943 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.421165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:33983 (local address 127.30.194.62:44619)
0808 17:01:47.421212 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.421215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.421401 (+ 186us) server_negotiation.cc:407] Connection header received
0808 17:01:47.421849 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.421850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.421865 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.421916 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:47.422507 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.422989 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.423903 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.424089 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.424144 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.426383 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:47.426385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:47.426385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:47.426392 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:47.427716 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.428118 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.428119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.428119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.428172 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.428410 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.428411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.428411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.428523 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:47.428576 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.428825 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.428883 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:01:47.429116 1534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.421132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:44619 (local address 127.30.194.3:33983)
0808 17:01:47.421294 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.421326 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.421756 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.421978 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.421982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.422234 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:47.422419 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.422424 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.423067 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.423068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.423823 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.423826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.424157 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.424158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.424274 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.424678 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:47.424687 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:47.426269 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:47.427785 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.427789 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.427791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.428048 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.428224 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.428225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.428225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.428351 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.428584 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:47.428589 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:47.428722 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.428803 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.428912 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260808 17:01:47.429467 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 1496
I20260808 17:01:47.429563 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:01:47.429756 31496 external_mini_cluster.cc:1428] Setting key e2ea234af9db1dd181b86b91bd09f524
I20260808 17:01:47.429890 1715 heartbeater.cc:347] Connected to a master server at 127.30.194.62:44619
I20260808 17:01:47.429972 1715 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:47.430156 1715 heartbeater.cc:511] Master 127.30.194.62:44619 requested a full tablet report, sending...
I20260808 17:01:47.430466 886 ts_manager.cc:194] Registered new tserver with Master: 83c6a3f2af4b4a4ca85f1dbb4902f1d0 (127.30.194.3:39383)
I20260808 17:01:47.430833 886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:33983
I20260808 17:01:47.435526 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.430539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:33825 (local address 127.30.194.62:44619)
0808 17:01:47.430578 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.430579 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.430679 (+ 100us) server_negotiation.cc:407] Connection header received
0808 17:01:47.430728 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.430729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.430739 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.430790 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:47.431091 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.431752 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.432448 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.432640 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.432691 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.433456 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:47.433457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:47.433458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:47.433464 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:47.434606 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.434892 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.434893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.434893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.434943 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.435099 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.435100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.435100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.435198 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:47.435257 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.435381 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.435442 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:01:47.435534 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.430469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:44619 (local address 127.30.194.3:33825)
0808 17:01:47.430618 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.430627 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.430675 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.430795 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.430795 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.430975 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:47.431030 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.431032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.431839 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.431839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.432380 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.432382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.432696 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.432697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.432777 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.433127 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:47.433131 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:47.433381 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:47.434676 (+ 1295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.434677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.434677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.434835 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.434994 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.434995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.434996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.435051 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.435250 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:47.435251 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:47.435294 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.435354 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.435443 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260808 17:01:47.441712 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 17:01:47 dist-test-slave-2j7r krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208507, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 17:01:47 dist-test-slave-2j7r krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208507, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:47.463374 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.457255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.30.194.62:44619)
0808 17:01:47.457310 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.457311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.457321 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:01:47.457382 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.457382 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.457391 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.457439 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:47.457934 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.458417 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.459115 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.459312 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.459362 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.461464 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:47.461465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:47.461465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:47.461471 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:47.462484 (+ 1013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.462785 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.462785 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.462786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.462826 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.462950 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.462951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.462951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.463071 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:47.463145 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.463262 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.463317 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:01:47.464928 886 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:01:47.466655 886 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:47.472087 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.468528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:36615 (local address 127.0.0.1:37578)
0808 17:01:47.468897 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.468905 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.468950 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.469142 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.469146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.469148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:47.469207 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.469236 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.470148 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.470149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.471062 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.471065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.471784 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.471785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.471882 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.471898 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.471967 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.472020 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":5,"thread_start_us":49,"threads_started":1}
I20260808 17:01:47.472124 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.468723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37578 (local address 127.30.194.2:36615)
0808 17:01:47.468910 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.468915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.468946 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:01:47.469011 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.469014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.469030 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.469133 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:47.469326 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.470073 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.471139 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.471709 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.471772 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.471829 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.471987 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.472029 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":88,"threads_started":1}
I20260808 17:01:47.472872 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.468509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:39383 (local address 127.0.0.1:37810)
0808 17:01:47.468653 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.468690 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.468770 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.469328 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.469330 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.469332 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:47.469374 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.469376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.470630 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.470631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.471817 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.471819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.472636 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.472637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.472695 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.472699 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.472761 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.472797 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":34,"thread_start_us":45,"threads_started":1}
I20260808 17:01:47.472892 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.468727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.30.194.3:39383)
0808 17:01:47.469075 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.469079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.469106 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:01:47.469139 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.469141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.469154 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.469222 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:47.469454 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.470553 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.471887 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.472561 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.472625 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.472680 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.472776 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.472808 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":87,"threads_started":1}
I20260808 17:01:47.472977 1746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.468509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37785 (local address 127.0.0.1:56576)
0808 17:01:47.468891 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.468901 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.468968 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.469212 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.469213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.469215 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:47.469252 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.469253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.470628 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.470629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.471815 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.471818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.472751 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.472752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.472818 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.472822 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.472869 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.472908 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":47,"thread_start_us":49,"threads_started":1}
I20260808 17:01:47.473194 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.468671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56576 (local address 127.30.194.1:37785)
0808 17:01:47.468862 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.468866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.468975 (+ 109us) server_negotiation.cc:407] Connection header received
0808 17:01:47.469025 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.469028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.469042 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.469149 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:47.469385 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.470529 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.471915 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.472584 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.472645 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.472703 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.473042 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.473081 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":88,"threads_started":1}
I20260808 17:01:47.473069 1363 tablet_service.cc:1467] Processing CreateTablet for tablet 105c92dee7f54444a5cd26c6bbc902c1 (DEFAULT_TABLE table=test-table [id=aa550b1e735b4de2bc49aada913ca4aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:47.473472 1363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105c92dee7f54444a5cd26c6bbc902c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:47.473706 1606 tablet_service.cc:1467] Processing CreateTablet for tablet 105c92dee7f54444a5cd26c6bbc902c1 (DEFAULT_TABLE table=test-table [id=aa550b1e735b4de2bc49aada913ca4aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:47.473776 1122 tablet_service.cc:1467] Processing CreateTablet for tablet 105c92dee7f54444a5cd26c6bbc902c1 (DEFAULT_TABLE table=test-table [id=aa550b1e735b4de2bc49aada913ca4aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:47.474002 1606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105c92dee7f54444a5cd26c6bbc902c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:47.474107 1122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105c92dee7f54444a5cd26c6bbc902c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:47.476255 1752 tablet_bootstrap.cc:492] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Bootstrap starting.
I20260808 17:01:47.476989 1754 tablet_bootstrap.cc:492] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: Bootstrap starting.
I20260808 17:01:47.477171 1752 tablet_bootstrap.cc:654] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:47.477550 1752 log.cc:824] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:47.477625 1754 tablet_bootstrap.cc:654] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:47.477631 1755 tablet_bootstrap.cc:492] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: Bootstrap starting.
I20260808 17:01:47.477921 1754 log.cc:824] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:47.478214 1755 tablet_bootstrap.cc:654] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:47.478384 1752 tablet_bootstrap.cc:492] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: No bootstrap required, opened a new log
I20260808 17:01:47.478439 1752 ts_tablet_manager.cc:1397] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:01:47.478506 1755 log.cc:824] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:47.479110 1755 tablet_bootstrap.cc:492] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: No bootstrap required, opened a new log
I20260808 17:01:47.479166 1755 ts_tablet_manager.cc:1397] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:01:47.479800 1752 raft_consensus.cc:348] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } }
I20260808 17:01:47.479920 1752 raft_consensus.cc:374] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:47.479940 1752 raft_consensus.cc:729] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c6a3f2af4b4a4ca85f1dbb4902f1d0, State: Initialized, Role: FOLLOWER
I20260808 17:01:47.480057 1752 consensus_queue.cc:260] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } }
I20260808 17:01:47.480300 1752 ts_tablet_manager.cc:1428] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:01:47.480444 1715 heartbeater.cc:503] Master 127.30.194.62:44619 was elected leader, sending a full tablet report...
I20260808 17:01:47.480713 1755 raft_consensus.cc:348] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } }
I20260808 17:01:47.480876 1755 raft_consensus.cc:374] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:47.480919 1755 raft_consensus.cc:729] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a124eec79334414ab930fdfb832aa7a1, State: Initialized, Role: FOLLOWER
I20260808 17:01:47.480930 1754 tablet_bootstrap.cc:492] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: No bootstrap required, opened a new log
I20260808 17:01:47.480993 1754 ts_tablet_manager.cc:1397] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 17:01:47.481037 1755 consensus_queue.cc:260] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } }
I20260808 17:01:47.481314 1755 ts_tablet_manager.cc:1428] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:01:47.481423 1224 heartbeater.cc:503] Master 127.30.194.62:44619 was elected leader, sending a full tablet report...
I20260808 17:01:47.482682 1754 raft_consensus.cc:348] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } }
I20260808 17:01:47.482800 1754 raft_consensus.cc:374] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:47.482831 1754 raft_consensus.cc:729] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea1ce95654c84b6689ff45b263f8b787, State: Initialized, Role: FOLLOWER
I20260808 17:01:47.482933 1754 consensus_queue.cc:260] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } }
I20260808 17:01:47.483153 1754 ts_tablet_manager.cc:1428] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 17:01:47.483222 1483 heartbeater.cc:503] Master 127.30.194.62:44619 was elected leader, sending a full tablet report...
W20260808 17:01:47.530110 1485 tablet.cc:2369] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:01:47.541294 1775 raft_consensus.cc:482] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:47.541406 1775 raft_consensus.cc:504] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } }
I20260808 17:01:47.541743 1775 leader_election.cc:288] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a124eec79334414ab930fdfb832aa7a1 (127.30.194.1:37785), 83c6a3f2af4b4a4ca85f1dbb4902f1d0 (127.30.194.3:39383)
I20260808 17:01:47.545154 1776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.541908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37785 (local address 127.30.194.2:48741)
0808 17:01:47.542223 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.542234 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.542297 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.542413 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.542414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.542415 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:47.542466 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.542467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.543264 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.543265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.544328 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.544329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.544878 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.544880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.544965 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.544969 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.545010 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.545045 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":138,"thread_start_us":119,"threads_started":1}
I20260808 17:01:47.545166 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.542034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:48741 (local address 127.30.194.1:37785)
0808 17:01:47.542082 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.542084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.542304 (+ 220us) server_negotiation.cc:407] Connection header received
0808 17:01:47.542340 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.542341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.542351 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.542410 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:47.542515 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.543183 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.544395 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.544790 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.544856 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.544864 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.545026 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.545076 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:01:47.545565 1155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105c92dee7f54444a5cd26c6bbc902c1" candidate_uuid: "ea1ce95654c84b6689ff45b263f8b787" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a124eec79334414ab930fdfb832aa7a1" is_pre_election: true
I20260808 17:01:47.545691 1291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.541908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:39383 (local address 127.30.194.2:46769)
0808 17:01:47.541949 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.541960 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.542000 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.542142 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.542143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.542145 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:47.542220 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.542222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.543135 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.543136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.544320 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.544323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.544910 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.544912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.545527 (+ 615us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.545532 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.545591 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.545631 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 17:01:47.545715 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.541963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:46769 (local address 127.30.194.3:39383)
0808 17:01:47.542024 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.542025 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.542034 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:47.542059 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.542060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.542070 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.542122 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:47.542297 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.543061 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.544441 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.544822 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.544895 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.544904 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.545608 (+ 704us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.545651 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:01:47.545729 1155 raft_consensus.cc:2454] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea1ce95654c84b6689ff45b263f8b787 in term 0.
I20260808 17:01:47.545962 1282 leader_election.cc:302] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a124eec79334414ab930fdfb832aa7a1, ea1ce95654c84b6689ff45b263f8b787; no voters:
I20260808 17:01:47.546098 1775 raft_consensus.cc:2781] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:47.546171 1775 raft_consensus.cc:482] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:47.546093 1639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105c92dee7f54444a5cd26c6bbc902c1" candidate_uuid: "ea1ce95654c84b6689ff45b263f8b787" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" is_pre_election: true
I20260808 17:01:47.546204 1775 raft_consensus.cc:3037] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:47.546264 1639 raft_consensus.cc:2454] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea1ce95654c84b6689ff45b263f8b787 in term 0.
I20260808 17:01:47.546928 1775 raft_consensus.cc:504] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } }
I20260808 17:01:47.547039 1775 leader_election.cc:288] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [CANDIDATE]: Term 1 election: Requested vote from peers a124eec79334414ab930fdfb832aa7a1 (127.30.194.1:37785), 83c6a3f2af4b4a4ca85f1dbb4902f1d0 (127.30.194.3:39383)
I20260808 17:01:47.547221 1155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105c92dee7f54444a5cd26c6bbc902c1" candidate_uuid: "ea1ce95654c84b6689ff45b263f8b787" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a124eec79334414ab930fdfb832aa7a1"
I20260808 17:01:47.547225 1639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105c92dee7f54444a5cd26c6bbc902c1" candidate_uuid: "ea1ce95654c84b6689ff45b263f8b787" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0"
I20260808 17:01:47.547297 1155 raft_consensus.cc:3037] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:47.547297 1639 raft_consensus.cc:3037] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:47.547982 1639 raft_consensus.cc:2454] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea1ce95654c84b6689ff45b263f8b787 in term 1.
I20260808 17:01:47.548018 1155 raft_consensus.cc:2454] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea1ce95654c84b6689ff45b263f8b787 in term 1.
I20260808 17:01:47.548159 1279 leader_election.cc:302] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83c6a3f2af4b4a4ca85f1dbb4902f1d0, ea1ce95654c84b6689ff45b263f8b787; no voters:
I20260808 17:01:47.548269 1775 raft_consensus.cc:2781] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:47.548398 1775 raft_consensus.cc:686] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 1 LEADER]: Becoming Leader. State: Replica: ea1ce95654c84b6689ff45b263f8b787, State: Running, Role: LEADER
I20260808 17:01:47.548481 1775 consensus_queue.cc:237] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } }
I20260808 17:01:47.549245 886 catalog_manager.cc:5495] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea1ce95654c84b6689ff45b263f8b787 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "ea1ce95654c84b6689ff45b263f8b787" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } health_report { overall_health: HEALTHY } } }
I20260808 17:01:47.567701 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.565555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.30.194.2:36615)
0808 17:01:47.565595 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.565597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.565637 (+ 40us) server_negotiation.cc:407] Connection header received
0808 17:01:47.565721 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.565721 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.565742 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.565797 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:47.565907 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.566406 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.567113 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.567285 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.567324 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.567465 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:47.567548 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:47.567597 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.567624 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.567639 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:01:47.569181 1639 raft_consensus.cc:1261] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 1 FOLLOWER]: Refusing update from remote peer ea1ce95654c84b6689ff45b263f8b787: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:47.569193 1155 raft_consensus.cc:1261] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1 [term 1 FOLLOWER]: Refusing update from remote peer ea1ce95654c84b6689ff45b263f8b787: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:47.569491 1778 consensus_queue.cc:1035] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:47.569609 1775 consensus_queue.cc:1035] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:47.572110 1785 mvcc.cc:205] Tried to move back new op lower bound from 7316310047001079808 to 7316310046919061504. Current Snapshot: MvccSnapshot[applied={T|T < 7316310047001079808}]
I20260808 17:01:47.572687 1784 mvcc.cc:205] Tried to move back new op lower bound from 7316310047001079808 to 7316310046919061504. Current Snapshot: MvccSnapshot[applied={T|T < 7316310047001079808}]
I20260808 17:01:47.575466 887 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:47.575662 887 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:47.576488 887 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac: Sending DeleteTablet for 3 replicas of tablet 105c92dee7f54444a5cd26c6bbc902c1
I20260808 17:01:47.576825 1122 tablet_service.cc:1514] Processing DeleteTablet for tablet 105c92dee7f54444a5cd26c6bbc902c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:47 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:56576
I20260808 17:01:47.576834 1363 tablet_service.cc:1514] Processing DeleteTablet for tablet 105c92dee7f54444a5cd26c6bbc902c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:47 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:37578
I20260808 17:01:47.576831 1606 tablet_service.cc:1514] Processing DeleteTablet for tablet 105c92dee7f54444a5cd26c6bbc902c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:47 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:37810
I20260808 17:01:47.577049 1796 tablet_replica.cc:330] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: stopping tablet replica
I20260808 17:01:47.577159 1796 raft_consensus.cc:2229] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:01:47.577242 1796 raft_consensus.cc:2258] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:47.577384 1797 tablet_replica.cc:330] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: stopping tablet replica
I20260808 17:01:47.577371 887 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 17:01:47.577472 1797 raft_consensus.cc:2229] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 17:01:47.577526 887 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:47.577564 1797 raft_consensus.cc:2258] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:47.578006 1796 ts_tablet_manager.cc:1905] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:47.579308 1122 tablet_service.cc:1467] Processing CreateTablet for tablet 4c56f33b438e4d90b26d7a508b32e29e (DEFAULT_TABLE table=test-table [id=195071927f304c53904c5b7c53ee357f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:47.579427 1122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c56f33b438e4d90b26d7a508b32e29e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:47.579586 1796 ts_tablet_manager.cc:1918] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:47.579645 1796 log.cc:1196] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/105c92dee7f54444a5cd26c6bbc902c1
I20260808 17:01:47.579891 1796 ts_tablet_manager.cc:1939] T 105c92dee7f54444a5cd26c6bbc902c1 P ea1ce95654c84b6689ff45b263f8b787: Deleting consensus metadata
I20260808 17:01:47.580338 1755 tablet_bootstrap.cc:492] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: Bootstrap starting.
I20260808 17:01:47.580480 862 catalog_manager.cc:4841] TS ea1ce95654c84b6689ff45b263f8b787 (127.30.194.2:36615): tablet 105c92dee7f54444a5cd26c6bbc902c1 (table test-table [id=aa550b1e735b4de2bc49aada913ca4aa]) successfully deleted
I20260808 17:01:47.577167 1798 tablet_replica.cc:330] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: stopping tablet replica
I20260808 17:01:47.580603 1798 raft_consensus.cc:2229] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:47.580690 1798 raft_consensus.cc:2258] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:47.580726 1755 tablet_bootstrap.cc:654] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:47.581049 1798 ts_tablet_manager.cc:1905] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:47.581156 1363 tablet_service.cc:1467] Processing CreateTablet for tablet 4c56f33b438e4d90b26d7a508b32e29e (DEFAULT_TABLE table=test-table [id=195071927f304c53904c5b7c53ee357f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:47.581246 1363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c56f33b438e4d90b26d7a508b32e29e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:47.581887 1606 tablet_service.cc:1467] Processing CreateTablet for tablet 4c56f33b438e4d90b26d7a508b32e29e (DEFAULT_TABLE table=test-table [id=195071927f304c53904c5b7c53ee357f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:47.582000 1606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c56f33b438e4d90b26d7a508b32e29e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:47.582296 1798 ts_tablet_manager.cc:1918] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:47.582356 1798 log.cc:1196] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/105c92dee7f54444a5cd26c6bbc902c1
I20260808 17:01:47.582352 1754 tablet_bootstrap.cc:492] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787: Bootstrap starting.
I20260808 17:01:47.582618 1798 ts_tablet_manager.cc:1939] T 105c92dee7f54444a5cd26c6bbc902c1 P a124eec79334414ab930fdfb832aa7a1: Deleting consensus metadata
I20260808 17:01:47.582744 1754 tablet_bootstrap.cc:654] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:47.583003 866 catalog_manager.cc:4841] TS a124eec79334414ab930fdfb832aa7a1 (127.30.194.1:37785): tablet 105c92dee7f54444a5cd26c6bbc902c1 (table test-table [id=aa550b1e735b4de2bc49aada913ca4aa]) successfully deleted
I20260808 17:01:47.583372 1797 ts_tablet_manager.cc:1905] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:47.583796 1752 tablet_bootstrap.cc:492] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Bootstrap starting.
I20260808 17:01:47.584192 1752 tablet_bootstrap.cc:654] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:47.584775 1752 tablet_bootstrap.cc:492] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: No bootstrap required, opened a new log
I20260808 17:01:47.584822 1752 ts_tablet_manager.cc:1397] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:47.584834 1797 ts_tablet_manager.cc:1918] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:47.584951 1797 log.cc:1196] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/105c92dee7f54444a5cd26c6bbc902c1
I20260808 17:01:47.584990 1752 raft_consensus.cc:348] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.585064 1752 raft_consensus.cc:374] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:47.585251 1752 raft_consensus.cc:729] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c6a3f2af4b4a4ca85f1dbb4902f1d0, State: Initialized, Role: FOLLOWER
I20260808 17:01:47.585318 1752 consensus_queue.cc:260] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.585477 1752 ts_tablet_manager.cc:1428] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:47.585595 1754 tablet_bootstrap.cc:492] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787: No bootstrap required, opened a new log
I20260808 17:01:47.585633 1754 ts_tablet_manager.cc:1397] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260808 17:01:47.585698 1755 tablet_bootstrap.cc:492] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: No bootstrap required, opened a new log
I20260808 17:01:47.585739 1755 ts_tablet_manager.cc:1397] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 17:01:47.585791 1754 raft_consensus.cc:348] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.585855 1754 raft_consensus.cc:374] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:47.585883 1754 raft_consensus.cc:729] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea1ce95654c84b6689ff45b263f8b787, State: Initialized, Role: FOLLOWER
I20260808 17:01:47.585882 1755 raft_consensus.cc:348] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.585954 1755 raft_consensus.cc:374] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:47.585924 1754 consensus_queue.cc:260] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.585974 1755 raft_consensus.cc:729] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a124eec79334414ab930fdfb832aa7a1, State: Initialized, Role: FOLLOWER
I20260808 17:01:47.586014 1755 consensus_queue.cc:260] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.586025 1754 ts_tablet_manager.cc:1428] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:47.586087 1755 ts_tablet_manager.cc:1428] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:47.585206 1797 ts_tablet_manager.cc:1939] T 105c92dee7f54444a5cd26c6bbc902c1 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Deleting consensus metadata
I20260808 17:01:47.586504 863 catalog_manager.cc:4841] TS 83c6a3f2af4b4a4ca85f1dbb4902f1d0 (127.30.194.3:39383): tablet 105c92dee7f54444a5cd26c6bbc902c1 (table test-table [id=aa550b1e735b4de2bc49aada913ca4aa]) successfully deleted
W20260808 17:01:47.636940 1229 tablet.cc:2369] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:01:47.651228 1792 raft_consensus.cc:482] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:47.651309 1792 raft_consensus.cc:504] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.651475 1792 leader_election.cc:288] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a124eec79334414ab930fdfb832aa7a1 (127.30.194.1:37785), 83c6a3f2af4b4a4ca85f1dbb4902f1d0 (127.30.194.3:39383)
I20260808 17:01:47.651692 1639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c56f33b438e4d90b26d7a508b32e29e" candidate_uuid: "ea1ce95654c84b6689ff45b263f8b787" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" is_pre_election: true
I20260808 17:01:47.651691 1155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c56f33b438e4d90b26d7a508b32e29e" candidate_uuid: "ea1ce95654c84b6689ff45b263f8b787" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a124eec79334414ab930fdfb832aa7a1" is_pre_election: true
I20260808 17:01:47.651796 1639 raft_consensus.cc:2454] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea1ce95654c84b6689ff45b263f8b787 in term 0.
I20260808 17:01:47.651813 1155 raft_consensus.cc:2454] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea1ce95654c84b6689ff45b263f8b787 in term 0.
I20260808 17:01:47.651993 1279 leader_election.cc:302] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83c6a3f2af4b4a4ca85f1dbb4902f1d0, ea1ce95654c84b6689ff45b263f8b787; no voters:
I20260808 17:01:47.652108 1792 raft_consensus.cc:2781] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:47.652158 1792 raft_consensus.cc:482] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:47.652177 1792 raft_consensus.cc:3037] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:47.652905 1792 raft_consensus.cc:504] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.653025 1792 leader_election.cc:288] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [CANDIDATE]: Term 1 election: Requested vote from peers a124eec79334414ab930fdfb832aa7a1 (127.30.194.1:37785), 83c6a3f2af4b4a4ca85f1dbb4902f1d0 (127.30.194.3:39383)
I20260808 17:01:47.653220 1155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c56f33b438e4d90b26d7a508b32e29e" candidate_uuid: "ea1ce95654c84b6689ff45b263f8b787" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a124eec79334414ab930fdfb832aa7a1"
I20260808 17:01:47.653219 1639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c56f33b438e4d90b26d7a508b32e29e" candidate_uuid: "ea1ce95654c84b6689ff45b263f8b787" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0"
I20260808 17:01:47.653301 1639 raft_consensus.cc:3037] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:47.653301 1155 raft_consensus.cc:3037] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:47.654009 1155 raft_consensus.cc:2454] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea1ce95654c84b6689ff45b263f8b787 in term 1.
I20260808 17:01:47.654009 1639 raft_consensus.cc:2454] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea1ce95654c84b6689ff45b263f8b787 in term 1.
I20260808 17:01:47.654174 1279 leader_election.cc:302] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83c6a3f2af4b4a4ca85f1dbb4902f1d0, ea1ce95654c84b6689ff45b263f8b787; no voters:
I20260808 17:01:47.654315 1792 raft_consensus.cc:2781] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:47.654366 1792 raft_consensus.cc:686] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 1 LEADER]: Becoming Leader. State: Replica: ea1ce95654c84b6689ff45b263f8b787, State: Running, Role: LEADER
I20260808 17:01:47.654415 1792 consensus_queue.cc:237] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.654958 887 catalog_manager.cc:5495] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea1ce95654c84b6689ff45b263f8b787 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "ea1ce95654c84b6689ff45b263f8b787" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } health_report { overall_health: UNKNOWN } } }
W20260808 17:01:47.678659 1716 tablet.cc:2369] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 17:01:47 dist-test-slave-2j7r krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208506, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:47.695614 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.689695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:44619 (local address 127.0.0.1:56550)
0808 17:01:47.689836 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:47.689845 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:47.689905 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:47.690015 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:47.690016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:47.690321 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:47.690371 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.690373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.690950 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.690950 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.691485 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:47.691486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.691751 (+ 265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.691751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:47.691787 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.692271 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:47.692275 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:47.693579 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:47.694771 (+ 1192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.694777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.694779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.694957 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.695093 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:47.695093 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:47.695093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:47.695154 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:47.695349 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:47.695354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:47.695392 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 17:01:47.695439 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:01:47.695523 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":95,"threads_started":1}
I20260808 17:01:47.695660 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:47.689764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56550 (local address 127.30.194.62:44619)
0808 17:01:47.689846 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:47.689848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:47.689893 (+ 45us) server_negotiation.cc:407] Connection header received
0808 17:01:47.689945 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:47.689946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:47.689954 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:47.690001 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:47.690430 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.690896 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.691538 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:47.691704 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:47.691745 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:47.693651 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:47.693652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:47.693652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:47.693658 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:47.694707 (+ 1049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.695010 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.695010 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.695011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.695052 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:47.695200 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:47.695200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:47.695200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:47.695306 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:47.695351 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:47.695460 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 17:01:47.695565 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":57}
I20260808 17:01:47.697592 887 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:56550:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 17:01:47.697785 887 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:47.699098 1606 tablet_service.cc:1467] Processing CreateTablet for tablet 55b750ee4c604c56a1f8dc93571db1f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6f6095b41cf14974939e6ff78baab449]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:01:47.699106 1363 tablet_service.cc:1467] Processing CreateTablet for tablet 55b750ee4c604c56a1f8dc93571db1f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6f6095b41cf14974939e6ff78baab449]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:01:47.699119 1122 tablet_service.cc:1467] Processing CreateTablet for tablet 55b750ee4c604c56a1f8dc93571db1f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6f6095b41cf14974939e6ff78baab449]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:01:47.699256 1363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b750ee4c604c56a1f8dc93571db1f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:47.699257 1606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b750ee4c604c56a1f8dc93571db1f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:47.699281 1122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b750ee4c604c56a1f8dc93571db1f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:47.701438 1754 tablet_bootstrap.cc:492] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: Bootstrap starting.
I20260808 17:01:47.701705 1754 tablet_bootstrap.cc:654] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:47.702172 1754 tablet_bootstrap.cc:492] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: No bootstrap required, opened a new log
I20260808 17:01:47.702210 1754 ts_tablet_manager.cc:1397] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 17:01:47.702302 1754 raft_consensus.cc:348] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.702353 1754 raft_consensus.cc:374] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:47.702369 1754 raft_consensus.cc:729] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea1ce95654c84b6689ff45b263f8b787, State: Initialized, Role: FOLLOWER
I20260808 17:01:47.702397 1754 consensus_queue.cc:260] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.702456 1754 ts_tablet_manager.cc:1428] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:47.702531 1792 tablet_replica.cc:439] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } }
I20260808 17:01:47.702572 1792 tablet_replica.cc:442] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:47.703500 1752 tablet_bootstrap.cc:492] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Bootstrap starting.
I20260808 17:01:47.703572 1755 tablet_bootstrap.cc:492] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: Bootstrap starting.
I20260808 17:01:47.703889 1752 tablet_bootstrap.cc:654] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:47.703903 1755 tablet_bootstrap.cc:654] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:47.704459 1755 tablet_bootstrap.cc:492] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: No bootstrap required, opened a new log
I20260808 17:01:47.704484 1752 tablet_bootstrap.cc:492] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: No bootstrap required, opened a new log
I20260808 17:01:47.704501 1755 ts_tablet_manager.cc:1397] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:47.704525 1752 ts_tablet_manager.cc:1397] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:47.704624 1755 raft_consensus.cc:348] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.704680 1755 raft_consensus.cc:374] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:47.704694 1755 raft_consensus.cc:729] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a124eec79334414ab930fdfb832aa7a1, State: Initialized, Role: FOLLOWER
I20260808 17:01:47.704656 1752 raft_consensus.cc:348] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.704736 1752 raft_consensus.cc:374] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:47.704725 1755 consensus_queue.cc:260] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.704764 1752 raft_consensus.cc:729] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c6a3f2af4b4a4ca85f1dbb4902f1d0, State: Initialized, Role: FOLLOWER
I20260808 17:01:47.704813 1755 ts_tablet_manager.cc:1428] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:47.704803 1752 consensus_queue.cc:260] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:47.704885 1752 ts_tablet_manager.cc:1428] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:47.704873 1774 tablet_replica.cc:439] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } }
I20260808 17:01:47.704947 1774 tablet_replica.cc:442] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:47.705030 1773 tablet_replica.cc:439] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } }
I20260808 17:01:47.705142 1773 tablet_replica.cc:442] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:48.080504 1792 consensus_queue.cc:1035] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:48.082084 1792 consensus_queue.cc:1035] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:48.110013 1774 raft_consensus.cc:482] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:48.110102 1774 raft_consensus.cc:504] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:48.110414 1774 leader_election.cc:288] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea1ce95654c84b6689ff45b263f8b787 (127.30.194.2:36615), 83c6a3f2af4b4a4ca85f1dbb4902f1d0 (127.30.194.3:39383)
I20260808 17:01:48.113262 1828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.110627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:49559 (local address 127.30.194.2:36615)
0808 17:01:48.110726 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:48.110728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:48.110918 (+ 190us) server_negotiation.cc:407] Connection header received
0808 17:01:48.110984 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:48.110985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:48.110995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:48.111062 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:48.111165 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.111644 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.112544 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.112924 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.112987 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.112997 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:48.113151 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 17:01:48.113193 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":51,"threads_started":1}
I20260808 17:01:48.113346 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.110901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:42645 (local address 127.30.194.3:39383)
0808 17:01:48.111059 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:48.111061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:48.111069 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:48.111092 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:48.111092 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:48.111102 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:48.111177 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:48.111317 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.111781 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.112614 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.112997 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.113056 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.113065 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:48.113238 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 17:01:48.113275 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":62,"threads_started":1}
I20260808 17:01:48.113372 1827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.110574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:39383 (local address 127.30.194.1:42645)
0808 17:01:48.110667 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:48.110680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:48.110745 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:48.111194 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:48.111195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:48.111197 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:48.111259 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:48.111260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.111850 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.111851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:48.112540 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:48.112542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.113096 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.113097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:48.113166 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.113170 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:48.113234 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:01:48.113271 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"thread_start_us":50,"threads_started":1}
I20260808 17:01:48.113262 1829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.110786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:36615 (local address 127.30.194.1:49559)
0808 17:01:48.110871 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:48.110878 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:48.110923 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:48.111064 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:48.111065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:48.111066 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:48.111119 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:48.111121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.111711 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.111712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:48.112465 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:48.112467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.112991 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.112992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:48.113065 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.113068 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:48.113144 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 17:01:48.113184 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":60,"threads_started":1}
I20260808 17:01:48.113547 1639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b750ee4c604c56a1f8dc93571db1f6" candidate_uuid: "a124eec79334414ab930fdfb832aa7a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" is_pre_election: true
I20260808 17:01:48.113618 1639 raft_consensus.cc:2454] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a124eec79334414ab930fdfb832aa7a1 in term 0.
I20260808 17:01:48.113803 1405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b750ee4c604c56a1f8dc93571db1f6" candidate_uuid: "a124eec79334414ab930fdfb832aa7a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea1ce95654c84b6689ff45b263f8b787" is_pre_election: true
I20260808 17:01:48.113914 1405 raft_consensus.cc:2454] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a124eec79334414ab930fdfb832aa7a1 in term 0.
I20260808 17:01:48.113893 1041 leader_election.cc:302] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83c6a3f2af4b4a4ca85f1dbb4902f1d0, a124eec79334414ab930fdfb832aa7a1; no voters:
I20260808 17:01:48.114027 1774 raft_consensus.cc:2781] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:48.114063 1774 raft_consensus.cc:482] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:48.114080 1774 raft_consensus.cc:3037] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:48.114715 1774 raft_consensus.cc:504] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:48.114820 1774 leader_election.cc:288] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [CANDIDATE]: Term 1 election: Requested vote from peers ea1ce95654c84b6689ff45b263f8b787 (127.30.194.2:36615), 83c6a3f2af4b4a4ca85f1dbb4902f1d0 (127.30.194.3:39383)
I20260808 17:01:48.114979 1405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b750ee4c604c56a1f8dc93571db1f6" candidate_uuid: "a124eec79334414ab930fdfb832aa7a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea1ce95654c84b6689ff45b263f8b787"
I20260808 17:01:48.115048 1405 raft_consensus.cc:3037] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:48.115015 1639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b750ee4c604c56a1f8dc93571db1f6" candidate_uuid: "a124eec79334414ab930fdfb832aa7a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0"
I20260808 17:01:48.115077 1639 raft_consensus.cc:3037] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:48.115653 1405 raft_consensus.cc:2454] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a124eec79334414ab930fdfb832aa7a1 in term 1.
I20260808 17:01:48.115660 1639 raft_consensus.cc:2454] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a124eec79334414ab930fdfb832aa7a1 in term 1.
I20260808 17:01:48.115823 1040 leader_election.cc:302] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a124eec79334414ab930fdfb832aa7a1, ea1ce95654c84b6689ff45b263f8b787; no voters:
I20260808 17:01:48.115916 1774 raft_consensus.cc:2781] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:48.116010 1774 raft_consensus.cc:686] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [term 1 LEADER]: Becoming Leader. State: Replica: a124eec79334414ab930fdfb832aa7a1, State: Running, Role: LEADER
I20260808 17:01:48.116062 1774 consensus_queue.cc:237] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } }
I20260808 17:01:48.116384 1832 tablet_replica.cc:439] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: TxnStatusTablet state changed. Reason: New leader a124eec79334414ab930fdfb832aa7a1. Latest consensus state: current_term: 1 leader_uuid: "a124eec79334414ab930fdfb832aa7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } }
I20260808 17:01:48.116470 1832 tablet_replica.cc:442] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:01:48.116657 1837 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 17:01:48.116761 1837 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 17:01:48.116696 886 catalog_manager.cc:5495] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a124eec79334414ab930fdfb832aa7a1 (127.30.194.1). New cstate: current_term: 1 leader_uuid: "a124eec79334414ab930fdfb832aa7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } health_report { overall_health: UNKNOWN } } }
I20260808 17:01:48.142653 1843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.139937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56586 (local address 127.30.194.1:37785)
0808 17:01:48.140077 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:48.140079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:48.140088 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:48.140122 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:48.140122 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:48.140138 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:48.140212 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:48.140334 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.140826 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.141928 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.142129 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.142180 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.142365 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:48.142463 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:48.142516 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:48.142577 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 17:01:48.142595 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260808 17:01:48.142694 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.139897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37785 (local address 127.0.0.1:56586)
0808 17:01:48.139976 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:48.139987 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:48.140026 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:48.140227 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:48.140228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:48.140229 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:01:48.140278 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:48.140281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.140898 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.140898 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:48.141842 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:48.141845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.142194 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.142195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:48.142262 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.142278 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:01:48.142518 (+ 240us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:01:48.142524 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:01:48.142571 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:01:48.142617 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 17:01:48.143837 1639 raft_consensus.cc:1261] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 1 FOLLOWER]: Refusing update from remote peer a124eec79334414ab930fdfb832aa7a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:48.143944 1405 raft_consensus.cc:1261] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787 [term 1 FOLLOWER]: Refusing update from remote peer a124eec79334414ab930fdfb832aa7a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:48.144109 1832 consensus_queue.cc:1035] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:48.144224 1774 consensus_queue.cc:1035] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:48.144694 1773 tablet_replica.cc:439] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: TxnStatusTablet state changed. Reason: New leader a124eec79334414ab930fdfb832aa7a1. Latest consensus state: current_term: 1 leader_uuid: "a124eec79334414ab930fdfb832aa7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } }
I20260808 17:01:48.144788 1773 tablet_replica.cc:442] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:48.144865 1792 tablet_replica.cc:439] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: TxnStatusTablet state changed. Reason: New leader a124eec79334414ab930fdfb832aa7a1. Latest consensus state: current_term: 1 leader_uuid: "a124eec79334414ab930fdfb832aa7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } }
I20260808 17:01:48.144966 1792 tablet_replica.cc:442] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:48.145870 1832 tablet_replica.cc:439] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a124eec79334414ab930fdfb832aa7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } }
I20260808 17:01:48.145984 1832 tablet_replica.cc:442] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:01:48.145869 1774 tablet_replica.cc:439] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a124eec79334414ab930fdfb832aa7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } }
I20260808 17:01:48.146085 1774 tablet_replica.cc:442] T 55b750ee4c604c56a1f8dc93571db1f6 P a124eec79334414ab930fdfb832aa7a1: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:01:48.146915 1773 tablet_replica.cc:439] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a124eec79334414ab930fdfb832aa7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } }
I20260808 17:01:48.146997 1773 tablet_replica.cc:442] T 55b750ee4c604c56a1f8dc93571db1f6 P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:48.147353 1820 mvcc.cc:205] Tried to move back new op lower bound from 7316310049355300864 to 7316310049243885568. Current Snapshot: MvccSnapshot[applied={T|T < 7316310049355300864}]
I20260808 17:01:48.147496 1792 tablet_replica.cc:439] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a124eec79334414ab930fdfb832aa7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a124eec79334414ab930fdfb832aa7a1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37785 } } peers { permanent_uuid: "ea1ce95654c84b6689ff45b263f8b787" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36615 } } peers { permanent_uuid: "83c6a3f2af4b4a4ca85f1dbb4902f1d0" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39383 } } }
I20260808 17:01:48.147583 1792 tablet_replica.cc:442] T 55b750ee4c604c56a1f8dc93571db1f6 P ea1ce95654c84b6689ff45b263f8b787: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:48.152227 1843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.149933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:44417 (local address 127.30.194.1:37785)
0808 17:01:48.149973 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:48.149976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:48.150096 (+ 120us) server_negotiation.cc:407] Connection header received
0808 17:01:48.150163 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:48.150164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:48.150175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:48.150256 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:48.150374 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.150814 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.151545 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.151752 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.151814 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.151988 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:48.152022 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:48.152066 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:48.152132 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 17:01:48.152155 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:01:48.152252 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.149857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37785 (local address 127.30.194.2:44417)
0808 17:01:48.150029 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:48.150039 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:48.150101 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:48.150265 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:48.150266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:48.150268 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:01:48.150317 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:48.150318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.150890 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.150891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:48.151466 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:48.151468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.151820 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.151821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:48.151893 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.151907 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:01:48.152073 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:01:48.152075 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:01:48.152128 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 17:01:48.152178 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260808 17:01:48.161507 1828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.159060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:53035 (local address 127.30.194.2:36615)
0808 17:01:48.159104 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:48.159106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:48.159113 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:01:48.159134 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:48.159135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:48.159146 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:48.159212 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:48.159346 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.159868 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.160633 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.160824 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.160884 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.161198 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:48.161229 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:48.161279 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:48.161423 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 17:01:48.161445 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:01:48.161512 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.158789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:36615 (local address 127.30.194.1:53035)
0808 17:01:48.158950 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:48.158961 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:48.159007 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:48.159218 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:48.159219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:48.159221 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:01:48.159284 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:48.159285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.159949 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.159950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:48.160552 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:48.160554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.160970 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.160972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:48.161036 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.161053 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:01:48.161306 (+ 253us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:01:48.161309 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:48.161359 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:01:48.161405 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260808 17:01:48.174157 886 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:48.174252 886 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:48.174841 886 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f73ca68920c849c99e7a19f2163242ac: Sending DeleteTablet for 3 replicas of tablet 4c56f33b438e4d90b26d7a508b32e29e
I20260808 17:01:48.175117 1606 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c56f33b438e4d90b26d7a508b32e29e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:48 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:37810
I20260808 17:01:48.175143 1122 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c56f33b438e4d90b26d7a508b32e29e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:48 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:56576
I20260808 17:01:48.175282 1869 tablet_replica.cc:330] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: stopping tablet replica
I20260808 17:01:48.175341 1871 tablet_replica.cc:330] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: stopping tablet replica
I20260808 17:01:48.175359 1869 raft_consensus.cc:2229] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:48.175398 1871 raft_consensus.cc:2229] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:48.175442 1871 raft_consensus.cc:2258] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:48.175448 1869 raft_consensus.cc:2258] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:48.175557 1363 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c56f33b438e4d90b26d7a508b32e29e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:48 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:37578
I20260808 17:01:48.175827 1871 ts_tablet_manager.cc:1905] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:48.175843 1869 ts_tablet_manager.cc:1905] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:48.177547 1869 ts_tablet_manager.cc:1918] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:01:48.177547 1871 ts_tablet_manager.cc:1918] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:01:48.177609 1871 log.cc:1196] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/4c56f33b438e4d90b26d7a508b32e29e
I20260808 17:01:48.177718 1869 log.cc:1196] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/4c56f33b438e4d90b26d7a508b32e29e
I20260808 17:01:48.177903 1871 ts_tablet_manager.cc:1939] T 4c56f33b438e4d90b26d7a508b32e29e P a124eec79334414ab930fdfb832aa7a1: Deleting consensus metadata
I20260808 17:01:48.178040 1869 ts_tablet_manager.cc:1939] T 4c56f33b438e4d90b26d7a508b32e29e P 83c6a3f2af4b4a4ca85f1dbb4902f1d0: Deleting consensus metadata
I20260808 17:01:48.178211 866 catalog_manager.cc:4841] TS a124eec79334414ab930fdfb832aa7a1 (127.30.194.1:37785): tablet 4c56f33b438e4d90b26d7a508b32e29e (table test-table [id=195071927f304c53904c5b7c53ee357f]) successfully deleted
I20260808 17:01:48.178421 1872 tablet_replica.cc:330] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787: stopping tablet replica
I20260808 17:01:48.178458 863 catalog_manager.cc:4841] TS 83c6a3f2af4b4a4ca85f1dbb4902f1d0 (127.30.194.3:39383): tablet 4c56f33b438e4d90b26d7a508b32e29e (table test-table [id=195071927f304c53904c5b7c53ee357f]) successfully deleted
I20260808 17:01:48.178561 1872 raft_consensus.cc:2229] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:01:48.178634 1872 raft_consensus.cc:2258] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:48.178848 1872 ts_tablet_manager.cc:1905] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:48.180403 1872 ts_tablet_manager.cc:1918] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:01:48.180459 1872 log.cc:1196] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/4c56f33b438e4d90b26d7a508b32e29e
I20260808 17:01:48.180684 1872 ts_tablet_manager.cc:1939] T 4c56f33b438e4d90b26d7a508b32e29e P ea1ce95654c84b6689ff45b263f8b787: Deleting consensus metadata
I20260808 17:01:48.181000 862 catalog_manager.cc:4841] TS ea1ce95654c84b6689ff45b263f8b787 (127.30.194.2:36615): tablet 4c56f33b438e4d90b26d7a508b32e29e (table test-table [id=195071927f304c53904c5b7c53ee357f]) successfully deleted
Aug 08 17:01:48 dist-test-slave-2j7r krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208507, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:01:48.184919 1843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.176405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.30.194.1:37785)
0808 17:01:48.176442 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:48.176443 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:48.176624 (+ 181us) server_negotiation.cc:407] Connection header received
0808 17:01:48.176708 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:48.176708 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:48.176732 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:48.176759 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:48.177470 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.178165 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.179198 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.179447 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.179530 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.182209 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:48.182216 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:48.182216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:48.182223 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:48.183764 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:48.184181 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:48.184184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:48.184185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:48.184248 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:48.184445 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:48.184445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:48.184446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:48.184559 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:48.184666 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:48.184799 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 17:01:48.184863 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 17:01:48.185156 1216 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56602
I20260808 17:01:48.191897 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:48.185964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56558 (local address 127.30.194.62:44619)
0808 17:01:48.186009 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:48.186010 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:48.186023 (+ 13us) server_negotiation.cc:407] Connection header received
0808 17:01:48.186091 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:48.186091 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:48.186100 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:48.186150 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:48.186593 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.187317 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.187950 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:48.188237 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:48.188307 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:48.189395 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:48.189396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:48.189397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:48.189404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:48.190880 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:48.191202 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:48.191204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:48.191204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:48.191266 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:48.191431 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:48.191432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:48.191432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:48.191560 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:48.191657 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:48.191740 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 17:01:48.191818 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 17:01:48.192111 886 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56558
I20260808 17:01:48.192571 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 1004
I20260808 17:01:48.227464 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 1247
I20260808 17:01:48.238618 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 1496
I20260808 17:01:48.249287 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 832
2026-08-08T17:01:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2671 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:48 dist-test-slave-2j7r krb5kdc[1880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:48 dist-test-slave-2j7r krb5kdc[1880](info): set up 2 sockets
Aug 08 17:01:48 dist-test-slave-2j7r krb5kdc[1880](info): commencing operation
krb5kdc: starting...
W20260808 17:01:49.391460 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:01:49 dist-test-slave-2j7r krb5kdc[1880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:01:49.505338 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:36661
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:36661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:49.585839 1900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:49.586030 1900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:49.586050 1900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:49.587100 1900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:49.587132 1900 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 17:01:49.587144 1900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:49.587155 1900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:49.587167 1900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:49.587177 1900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:49.588541 1900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:36661
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:36661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:49.588711 1900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:49.588861 1900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:49.593364 1906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:49.593369 1909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:49.593433 1911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:49.593461 1900 server_base.cc:1034] running on GCE node
I20260808 17:01:49.593783 1900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:49.594069 1900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:49.595249 1900 hybrid_clock.cc:648] HybridClock initialized: now 1786208509595226 us; error 37 us; skew 500 ppm
Aug 08 17:01:49 dist-test-slave-2j7r krb5kdc[1880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208509, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:49.597589 1900 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:01:49.598004 1900 webserver.cc:460] Webserver started at http://127.30.194.62:42195/ using document root <none> and password file <none>
I20260808 17:01:49.598206 1900 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:49.598264 1900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:49.598399 1900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:49.599388 1900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "482478e7fe81458b8426da286003a3f7"
format_stamp: "Formatted at 2026-08-08 17:01:49 on dist-test-slave-2j7r"
server_key: "18889665a5a52fd320cb6370295010aa"
server_key_iv: "d081ce6561256bf55c0c0f409f6b522d"
server_key_version: "encryptionkey@0"
I20260808 17:01:49.599745 1900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "482478e7fe81458b8426da286003a3f7"
format_stamp: "Formatted at 2026-08-08 17:01:49 on dist-test-slave-2j7r"
server_key: "18889665a5a52fd320cb6370295010aa"
server_key_iv: "d081ce6561256bf55c0c0f409f6b522d"
server_key_version: "encryptionkey@0"
I20260808 17:01:49.601119 1900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:01:49.601907 1924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:49.602077 1900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:49.602159 1900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "482478e7fe81458b8426da286003a3f7"
format_stamp: "Formatted at 2026-08-08 17:01:49 on dist-test-slave-2j7r"
server_key: "18889665a5a52fd320cb6370295010aa"
server_key_iv: "d081ce6561256bf55c0c0f409f6b522d"
server_key_version: "encryptionkey@0"
I20260808 17:01:49.602226 1900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:49.612205 1900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:49.613903 1900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:49.614024 1900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:49.618251 1900 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:36661
I20260808 17:01:49.618319 2029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:36661 every 8 connection(s)
I20260808 17:01:49.618582 1900 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:49.619175 2030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:49.619985 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 1900
I20260808 17:01:49.620095 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:01:49.620405 31496 external_mini_cluster.cc:1428] Setting key 32a2bc4f8f8f05f90ae1495a037a3a80
I20260808 17:01:49.622851 2030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7: Bootstrap starting.
I20260808 17:01:49.623533 2030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:49.623816 2030 log.cc:824] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:49.624615 2030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7: No bootstrap required, opened a new log
I20260808 17:01:49.626976 2030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482478e7fe81458b8426da286003a3f7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36661 } }
I20260808 17:01:49.627137 2030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:49.627178 2030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482478e7fe81458b8426da286003a3f7, State: Initialized, Role: FOLLOWER
I20260808 17:01:49.627328 2030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482478e7fe81458b8426da286003a3f7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36661 } }
I20260808 17:01:49.627408 2030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:49.627440 2030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:49.627485 2030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 17:01:49 dist-test-slave-2j7r krb5kdc[1880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:49.628039 2030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482478e7fe81458b8426da286003a3f7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36661 } }
I20260808 17:01:49.628135 2030 leader_election.cc:302] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 482478e7fe81458b8426da286003a3f7; no voters:
I20260808 17:01:49.628309 2030 leader_election.cc:288] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:49.628403 2036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:49.628549 2036 raft_consensus.cc:686] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [term 1 LEADER]: Becoming Leader. State: Replica: 482478e7fe81458b8426da286003a3f7, State: Running, Role: LEADER
I20260808 17:01:49.628556 2030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:49.628783 2036 consensus_queue.cc:237] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482478e7fe81458b8426da286003a3f7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36661 } }
I20260808 17:01:49.629400 2039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 482478e7fe81458b8426da286003a3f7. Latest consensus state: current_term: 1 leader_uuid: "482478e7fe81458b8426da286003a3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482478e7fe81458b8426da286003a3f7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36661 } } }
I20260808 17:01:49.629477 2039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:49.629635 2037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "482478e7fe81458b8426da286003a3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482478e7fe81458b8426da286003a3f7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36661 } } }
I20260808 17:01:49.629684 2047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:49.629709 2037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:49.630188 2047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:49.631946 2047 catalog_manager.cc:1347] Generated new cluster ID: a89b31a668324814832c0b2dd5b3fa7f
I20260808 17:01:49.631992 2047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:49.632961 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:49.620967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.30.194.62:36661)
0808 17:01:49.621163 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:49.621172 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:49.621206 (+ 34us) server_negotiation.cc:407] Connection header received
0808 17:01:49.621703 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:49.621712 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:49.621784 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:49.621927 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:49.622645 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.623936 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.624964 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.625611 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.625727 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:49.628309 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:49.628313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:49.628313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:49.628325 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:49.630917 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:49.631308 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:49.631312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:49.631312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:49.631417 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:49.631623 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:49.631624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:49.631624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:49.631916 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:49.632050 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:49.632625 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 17:01:49.632750 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260808 17:01:49.646498 2047 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:49.647094 2047 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:49.652288 2047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7: Generated new TSK 0
I20260808 17:01:49.652447 2047 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:01:49.683257 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36661
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:49.762357 2071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:49.762544 2071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:49.762576 2071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:49.763659 2071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:49.763697 2071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:49.763723 2071 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 17:01:49.763738 2071 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:01:49.763761 2071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:01:49.765169 2071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.194.62:36661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:49.765398 2071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:49.765585 2071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:49.770172 2084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:49.770215 2082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:49.770179 2081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:49.770542 2071 server_base.cc:1034] running on GCE node
I20260808 17:01:49.770726 2071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:49.770917 2071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:49.772058 2071 hybrid_clock.cc:648] HybridClock initialized: now 1786208509772047 us; error 31 us; skew 500 ppm
Aug 08 17:01:49 dist-test-slave-2j7r krb5kdc[1880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208509, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:49.778769 2071 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:01:49.779161 2071 webserver.cc:460] Webserver started at http://127.30.194.1:38641/ using document root <none> and password file <none>
I20260808 17:01:49.779345 2071 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:49.779394 2071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:49.779517 2071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:49.780462 2071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "351afd7919474272bc4431d70efb0987"
format_stamp: "Formatted at 2026-08-08 17:01:49 on dist-test-slave-2j7r"
server_key: "f944d7d12728f1aa7be0fea243337074"
server_key_iv: "44d2ecb9103b3bbe7d9704b6f8a1fd80"
server_key_version: "encryptionkey@0"
I20260808 17:01:49.780768 2071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "351afd7919474272bc4431d70efb0987"
format_stamp: "Formatted at 2026-08-08 17:01:49 on dist-test-slave-2j7r"
server_key: "f944d7d12728f1aa7be0fea243337074"
server_key_iv: "44d2ecb9103b3bbe7d9704b6f8a1fd80"
server_key_version: "encryptionkey@0"
I20260808 17:01:49.782088 2071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:49.782910 2092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:49.783090 2071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:49.783181 2071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "351afd7919474272bc4431d70efb0987"
format_stamp: "Formatted at 2026-08-08 17:01:49 on dist-test-slave-2j7r"
server_key: "f944d7d12728f1aa7be0fea243337074"
server_key_iv: "44d2ecb9103b3bbe7d9704b6f8a1fd80"
server_key_version: "encryptionkey@0"
I20260808 17:01:49.783246 2071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:49.818643 2071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:49.820250 2071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:49.820374 2071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:49.820828 2071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:49.820869 2071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:49.820896 2071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:49.820912 2071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:01:49 dist-test-slave-2j7r krb5kdc[1880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208509, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:49.827462 2071 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:43947
I20260808 17:01:49.827517 2296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:43947 every 8 connection(s)
I20260808 17:01:49.827843 2071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:01:49.828016 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 2071
I20260808 17:01:49.828130 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:01:49.828603 31496 external_mini_cluster.cc:1428] Setting key d36efdfb0d02db8051cad48869195a5e
I20260808 17:01:49.830247 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:49.821255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:52365 (local address 127.30.194.62:36661)
0808 17:01:49.821301 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:49.821303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:49.821536 (+ 233us) server_negotiation.cc:407] Connection header received
0808 17:01:49.821954 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:49.821955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:49.821969 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:49.822042 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:49.822773 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.823510 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.824548 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.824859 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.824930 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:49.827404 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:49.827406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:49.827406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:49.827414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:49.828883 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:49.829361 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:49.829363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:49.829363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:49.829426 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:49.829639 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:49.829640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:49.829640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:49.829760 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:49.829814 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:49.830073 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 17:01:49.830157 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:01:49.830401 2107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:49.821270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36661 (local address 127.30.194.1:52365)
0808 17:01:49.821422 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:49.821455 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:49.821872 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:49.822090 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:49.822093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:49.822399 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:49.822664 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:49.822670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.823602 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.823603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:49.824464 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:49.824467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.824959 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.824961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:49.825131 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:49.825622 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:49.825643 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:49.827316 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:49.828946 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:49.828954 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:49.828956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:49.829294 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:49.829480 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:49.829481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:49.829481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:49.829585 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:49.829814 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:49.829821 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:49.829946 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 17:01:49.830043 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 17:01:49.830187 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260808 17:01:49.830922 2297 heartbeater.cc:347] Connected to a master server at 127.30.194.62:36661
I20260808 17:01:49.831001 2297 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:49.831163 2297 heartbeater.cc:511] Master 127.30.194.62:36661 requested a full tablet report, sending...
I20260808 17:01:49.831640 1953 ts_manager.cc:194] Registered new tserver with Master: 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947)
I20260808 17:01:49.832485 1953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:52365
I20260808 17:01:49.837061 2308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:49.831895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36661 (local address 127.30.194.1:51533)
0808 17:01:49.832020 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:49.832030 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:49.832100 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:49.832252 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:49.832252 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:49.832418 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:49.832470 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:49.832472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.833263 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.833263 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:49.833753 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:49.833755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.834096 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.834097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:49.834177 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:49.834535 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:49.834540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:49.834795 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:49.836247 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:49.836248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:49.836248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:49.836398 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:49.836578 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:49.836578 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:49.836579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:49.836622 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:49.836845 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:49.836846 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:49.836882 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 17:01:49.836931 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 17:01:49.837001 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260808 17:01:49.837248 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:49.831959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:51533 (local address 127.30.194.62:36661)
0808 17:01:49.831994 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:49.831996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:49.832077 (+ 81us) server_negotiation.cc:407] Connection header received
0808 17:01:49.832157 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:49.832159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:49.832169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:49.832194 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:49.832553 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.833135 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.833819 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.834020 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.834079 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:49.834871 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:49.834872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:49.834872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:49.834879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:49.836155 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:49.836463 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:49.836464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:49.836464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:49.836521 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:49.836658 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:49.836659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:49.836659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:49.836784 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:49.836846 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:49.836975 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 17:01:49.837047 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:01:49.853303 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36661
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:49.933034 2310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:49.933244 2310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:49.933267 2310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:49.934327 2310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:49.934360 2310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:49.934379 2310 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 17:01:49.934389 2310 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:01:49.934405 2310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:01:49.935773 2310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.194.62:36661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:49.935945 2310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:49.936115 2310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:49.940652 2316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:49.940724 2321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:49.940653 2317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:49.940811 2310 server_base.cc:1034] running on GCE node
I20260808 17:01:49.941108 2310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:49.941380 2310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:49.942516 2310 hybrid_clock.cc:648] HybridClock initialized: now 1786208509942508 us; error 31 us; skew 500 ppm
Aug 08 17:01:49 dist-test-slave-2j7r krb5kdc[1880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208509, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:49.944759 2310 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:01:49.945230 2310 webserver.cc:460] Webserver started at http://127.30.194.2:42981/ using document root <none> and password file <none>
I20260808 17:01:49.945430 2310 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:49.945477 2310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:49.945600 2310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:49.946552 2310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "754295157b374497891db284c0a93963"
format_stamp: "Formatted at 2026-08-08 17:01:49 on dist-test-slave-2j7r"
server_key: "4c8f1235f149dfa64eb715926a6443d3"
server_key_iv: "c95977c441e4c1f7fe020fbd2375a523"
server_key_version: "encryptionkey@0"
I20260808 17:01:49.946871 2310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "754295157b374497891db284c0a93963"
format_stamp: "Formatted at 2026-08-08 17:01:49 on dist-test-slave-2j7r"
server_key: "4c8f1235f149dfa64eb715926a6443d3"
server_key_iv: "c95977c441e4c1f7fe020fbd2375a523"
server_key_version: "encryptionkey@0"
I20260808 17:01:49.948068 2310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:49.948699 2334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:49.948834 2310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:49.948915 2310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "754295157b374497891db284c0a93963"
format_stamp: "Formatted at 2026-08-08 17:01:49 on dist-test-slave-2j7r"
server_key: "4c8f1235f149dfa64eb715926a6443d3"
server_key_iv: "c95977c441e4c1f7fe020fbd2375a523"
server_key_version: "encryptionkey@0"
I20260808 17:01:49.948980 2310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:49.957472 2310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:49.958994 2310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:49.959115 2310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:49.959595 2310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:49.959635 2310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:49.959662 2310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:49.959698 2310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:01:49 dist-test-slave-2j7r krb5kdc[1880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208509, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:49.966617 2310 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:41279
I20260808 17:01:49.966678 2515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:41279 every 8 connection(s)
I20260808 17:01:49.966971 2310 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:01:49.967471 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 2310
I20260808 17:01:49.967557 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:01:49.967734 31496 external_mini_cluster.cc:1428] Setting key 66a5381fdb63f58c649d3fb8404e69f9
I20260808 17:01:49.968468 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:49.959958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:40485 (local address 127.30.194.62:36661)
0808 17:01:49.960003 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:49.960005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:49.960187 (+ 182us) server_negotiation.cc:407] Connection header received
0808 17:01:49.960646 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:49.960648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:49.960661 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:49.960711 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:49.961376 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.961874 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.962810 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.962996 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.963050 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:49.965812 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:49.965813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:49.965814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:49.965822 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:49.967278 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:49.967627 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:49.967628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:49.967628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:49.967691 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:49.967929 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:49.967930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:49.967931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:49.968052 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:49.968117 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:49.968343 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 17:01:49.968406 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:01:49.968675 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:49.959953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36661 (local address 127.30.194.2:40485)
0808 17:01:49.960083 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:49.960114 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:49.960551 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:49.960772 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:49.960776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:49.961061 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:49.961287 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:49.961292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.961954 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.961955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:49.962724 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:49.962726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.963063 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.963064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:49.963194 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:49.963670 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:49.963679 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:49.965727 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:49.967358 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:49.967362 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:49.967363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:49.967564 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:49.967747 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:49.967748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:49.967749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:49.967861 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:49.968109 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:49.968114 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:49.968219 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 17:01:49.968320 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 17:01:49.968475 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":72,"threads_started":1}
I20260808 17:01:49.970345 2518 heartbeater.cc:347] Connected to a master server at 127.30.194.62:36661
I20260808 17:01:49.970439 2518 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:49.970606 2518 heartbeater.cc:511] Master 127.30.194.62:36661 requested a full tablet report, sending...
I20260808 17:01:49.970968 1954 ts_manager.cc:194] Registered new tserver with Master: 754295157b374497891db284c0a93963 (127.30.194.2:41279)
I20260808 17:01:49.971412 1954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:40485
I20260808 17:01:49.976648 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:49.971290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:46051 (local address 127.30.194.62:36661)
0808 17:01:49.971323 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:49.971325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:49.971424 (+ 99us) server_negotiation.cc:407] Connection header received
0808 17:01:49.971468 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:49.971468 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:49.971478 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:49.971521 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:49.971858 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.972417 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.973414 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.973585 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.973629 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:49.974454 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:49.974455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:49.974455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:49.974461 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:49.975664 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:49.976001 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:49.976001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:49.976002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:49.976042 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:49.976222 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:49.976222 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:49.976222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:49.976345 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:49.976392 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:49.976541 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 17:01:49.976594 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:01:49.976711 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:49.971253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36661 (local address 127.30.194.2:46051)
0808 17:01:49.971364 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:49.971373 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:49.971425 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:49.971549 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:49.971550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:49.971709 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:49.971784 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:49.971786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.972517 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.972518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:49.973334 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:49.973337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:49.973645 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:49.973646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:49.973717 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:49.974042 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:49.974046 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:49.974379 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:49.975734 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:49.975735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:49.975735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:49.975935 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:49.976091 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:49.976092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:49.976092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:49.976151 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:49.976404 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:49.976405 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:49.976450 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 17:01:49.976526 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 17:01:49.976614 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":55,"threads_started":1}
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:01:49.992022 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36661
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:50.070523 2535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:50.070694 2535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:50.070716 2535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:50.071830 2535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:50.071870 2535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:50.071889 2535 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 17:01:50.071899 2535 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:01:50.071914 2535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:01:50.073290 2535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.194.62:36661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:50.073454 2535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:50.073614 2535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:01:50.078318 2535 server_base.cc:1034] running on GCE node
W20260808 17:01:50.078248 2545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:50.078225 2542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:50.078236 2543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:50.078586 2535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:50.078802 2535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:50.079950 2535 hybrid_clock.cc:648] HybridClock initialized: now 1786208510079940 us; error 36 us; skew 500 ppm
Aug 08 17:01:50 dist-test-slave-2j7r krb5kdc[1880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208510, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:50.082252 2535 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:01:50.082664 2535 webserver.cc:460] Webserver started at http://127.30.194.3:45775/ using document root <none> and password file <none>
I20260808 17:01:50.082875 2535 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:50.082923 2535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:50.083042 2535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:50.083984 2535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "1249c6fa72ee4ac395cff1be557019a7"
format_stamp: "Formatted at 2026-08-08 17:01:50 on dist-test-slave-2j7r"
server_key: "93746463438fb4254ebb754d3b3f3542"
server_key_iv: "fe41c61cfecc9fd0e683e42174f5c279"
server_key_version: "encryptionkey@0"
I20260808 17:01:50.084303 2535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "1249c6fa72ee4ac395cff1be557019a7"
format_stamp: "Formatted at 2026-08-08 17:01:50 on dist-test-slave-2j7r"
server_key: "93746463438fb4254ebb754d3b3f3542"
server_key_iv: "fe41c61cfecc9fd0e683e42174f5c279"
server_key_version: "encryptionkey@0"
I20260808 17:01:50.085541 2535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:01:50.086278 2553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:50.086437 2535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:50.086513 2535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "1249c6fa72ee4ac395cff1be557019a7"
format_stamp: "Formatted at 2026-08-08 17:01:50 on dist-test-slave-2j7r"
server_key: "93746463438fb4254ebb754d3b3f3542"
server_key_iv: "fe41c61cfecc9fd0e683e42174f5c279"
server_key_version: "encryptionkey@0"
I20260808 17:01:50.086562 2535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:50.106449 2535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:50.108042 2535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:50.108142 2535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:50.108551 2535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:50.108592 2535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:50.108621 2535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:50.108637 2535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:50.115535 2535 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:44057
Aug 08 17:01:50 dist-test-slave-2j7r krb5kdc[1880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208510, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:50.115830 2770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:44057 every 8 connection(s)
I20260808 17:01:50.115911 2535 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:01:50.117385 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 2535
I20260808 17:01:50.117491 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:01:50.117686 31496 external_mini_cluster.cc:1428] Setting key b95e4e4969a59e0f64915f6711151f68
I20260808 17:01:50.119177 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.108991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:40523 (local address 127.30.194.62:36661)
0808 17:01:50.109059 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.109061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.109277 (+ 216us) server_negotiation.cc:407] Connection header received
0808 17:01:50.109877 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.109879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.109891 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.109953 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:50.110766 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.111489 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.112901 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.113208 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.113278 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.116246 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:50.116248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:50.116248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:50.116256 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:50.117879 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:50.118240 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:50.118241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:50.118241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:50.118311 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:50.118549 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:50.118550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:50.118551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:50.118653 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:50.118710 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.119001 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.119089 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:01:50.119302 2572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.108950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36661 (local address 127.30.194.3:40523)
0808 17:01:50.109144 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.109190 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.109755 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.110036 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.110041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.110385 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:50.110663 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.110669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.111573 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.111575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.112815 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.112819 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.113300 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.113301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.113446 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.114328 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:50.114338 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:50.116143 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:50.117952 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:50.117956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:50.117958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:50.118170 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:50.118358 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:50.118358 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:50.118359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:50.118487 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:50.118714 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:50.118721 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:50.118872 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.118966 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.119108 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
I20260808 17:01:50.119700 2771 heartbeater.cc:347] Connected to a master server at 127.30.194.62:36661
I20260808 17:01:50.119777 2771 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:50.119933 2771 heartbeater.cc:511] Master 127.30.194.62:36661 requested a full tablet report, sending...
I20260808 17:01:50.120287 1953 ts_manager.cc:194] Registered new tserver with Master: 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057)
I20260808 17:01:50.120748 1953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:40523
I20260808 17:01:50.125074 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.120582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36661 (local address 127.30.194.3:33189)
0808 17:01:50.120690 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.120699 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.120757 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.120921 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.120922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.121134 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:50.121205 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.121207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.121868 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.121869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.122341 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.122342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.122603 (+ 261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.122604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.122651 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.122936 (+ 285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:50.122939 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:50.123171 (+ 232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:50.124391 (+ 1220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:50.124391 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:50.124392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:50.124536 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:50.124677 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:50.124678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:50.124678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:50.124718 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:50.124880 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:50.124880 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:50.124911 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.124951 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.125015 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":6,"thread_start_us":60,"threads_started":1}
I20260808 17:01:50.125103 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.120660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:33189 (local address 127.30.194.62:36661)
0808 17:01:50.120695 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.120697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.120743 (+ 46us) server_negotiation.cc:407] Connection header received
0808 17:01:50.120808 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.120809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.120818 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.120868 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:50.121274 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.121796 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.122393 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.122556 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.122600 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.123266 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:50.123267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:50.123267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:50.123274 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:50.124329 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:50.124591 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:50.124592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:50.124592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:50.124639 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:50.124753 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:50.124753 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:50.124754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:50.124839 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:50.124885 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.124975 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.125027 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:01:50.129456 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 17:01:50 dist-test-slave-2j7r krb5kdc[1880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 17:01:50 dist-test-slave-2j7r krb5kdc[1880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:50.150695 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.144954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50554 (local address 127.30.194.62:36661)
0808 17:01:50.145002 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.145004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.145011 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:01:50.145048 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.145049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.145056 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.145121 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:50.145656 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.146109 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.146759 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.146928 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.146973 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.148815 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:50.148816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:50.148816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:50.148823 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:50.149919 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:50.150189 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:50.150189 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:50.150190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:50.150260 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:50.150381 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:50.150381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:50.150382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:50.150461 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:50.150523 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.150591 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.150639 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 17:01:50.152089 1953 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:01:50.153820 1953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:50.160050 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.155950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:43947 (local address 127.0.0.1:57054)
0808 17:01:50.156250 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.156317 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.156381 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.156625 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.156628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.156630 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:50.156686 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.156688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.158011 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.158011 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.159050 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.159052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.159750 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.159752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.159831 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.159847 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.159919 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.159973 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":1,"thread_start_us":45,"threads_started":1}
I20260808 17:01:50.160137 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.156128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.30.194.2:41279)
0808 17:01:50.156528 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.156532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.156542 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:01:50.156583 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.156586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.156602 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.156689 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:50.156864 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.157697 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.159109 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.159805 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.159867 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.159918 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.160008 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.160045 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":73,"threads_started":1}
I20260808 17:01:50.160235 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.156210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56840 (local address 127.30.194.3:44057)
0808 17:01:50.156368 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.156373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.156546 (+ 173us) server_negotiation.cc:407] Connection header received
0808 17:01:50.156584 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.156586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.156602 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.156695 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:50.156940 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.158009 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.159008 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.159718 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.159787 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.159843 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.160109 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.160144 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260808 17:01:50.160431 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.156061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57054 (local address 127.30.194.1:43947)
0808 17:01:50.156410 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.156414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.156425 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:01:50.156462 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.156464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.156479 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.156549 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:50.156778 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.157928 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.159144 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.159662 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.159739 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.159805 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.160274 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.160320 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":94,"threads_started":1}
I20260808 17:01:50.160485 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.155950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:41279 (local address 127.0.0.1:57730)
0808 17:01:50.156278 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.156320 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.156384 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.156734 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.156735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.156737 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:50.156777 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.156778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.157788 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.157790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.159008 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.159011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.159867 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.159868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.159925 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.159928 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.159992 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.160028 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260808 17:01:50.160601 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.156125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:44057 (local address 127.0.0.1:56840)
0808 17:01:50.156477 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.156485 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.156533 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.156846 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.156846 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.156847 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:50.156872 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.156873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.158066 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.158067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.158938 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.158941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.159880 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.159881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.159949 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.159953 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.159997 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.160039 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":65,"threads_started":1}
I20260808 17:01:50.161181 2614 tablet_service.cc:1467] Processing CreateTablet for tablet 60f515d7055d4a99902c1885210bcf37 (DEFAULT_TABLE table=test-table [id=7d5e50f4a16f45119d7795611353304a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:50.161181 2171 tablet_service.cc:1467] Processing CreateTablet for tablet 60f515d7055d4a99902c1885210bcf37 (DEFAULT_TABLE table=test-table [id=7d5e50f4a16f45119d7795611353304a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:50.161298 2411 tablet_service.cc:1467] Processing CreateTablet for tablet 60f515d7055d4a99902c1885210bcf37 (DEFAULT_TABLE table=test-table [id=7d5e50f4a16f45119d7795611353304a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:50.161533 2614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f515d7055d4a99902c1885210bcf37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:50.161533 2171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f515d7055d4a99902c1885210bcf37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:50.161581 2411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f515d7055d4a99902c1885210bcf37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:50.164283 2811 tablet_bootstrap.cc:492] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: Bootstrap starting.
I20260808 17:01:50.164774 2813 tablet_bootstrap.cc:492] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: Bootstrap starting.
I20260808 17:01:50.164901 2815 tablet_bootstrap.cc:492] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: Bootstrap starting.
I20260808 17:01:50.164971 2811 tablet_bootstrap.cc:654] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:50.165364 2811 log.cc:824] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:50.165582 2815 tablet_bootstrap.cc:654] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:50.165624 2813 tablet_bootstrap.cc:654] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:50.165946 2813 log.cc:824] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:50.165926 2815 log.cc:824] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:50.166240 2811 tablet_bootstrap.cc:492] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: No bootstrap required, opened a new log
I20260808 17:01:50.166302 2811 ts_tablet_manager.cc:1397] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:01:50.166702 2813 tablet_bootstrap.cc:492] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: No bootstrap required, opened a new log
I20260808 17:01:50.166736 2815 tablet_bootstrap.cc:492] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: No bootstrap required, opened a new log
I20260808 17:01:50.166781 2813 ts_tablet_manager.cc:1397] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:01:50.166800 2815 ts_tablet_manager.cc:1397] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:01:50.168251 2815 raft_consensus.cc:348] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.168378 2815 raft_consensus.cc:374] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:50.168409 2815 raft_consensus.cc:729] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351afd7919474272bc4431d70efb0987, State: Initialized, Role: FOLLOWER
I20260808 17:01:50.168433 2811 raft_consensus.cc:348] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.168538 2815 consensus_queue.cc:260] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.168604 2811 raft_consensus.cc:374] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:50.168653 2811 raft_consensus.cc:729] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754295157b374497891db284c0a93963, State: Initialized, Role: FOLLOWER
I20260808 17:01:50.168787 2815 ts_tablet_manager.cc:1428] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:01:50.168782 2811 consensus_queue.cc:260] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.168736 2813 raft_consensus.cc:348] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.168869 2813 raft_consensus.cc:374] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:50.168897 2297 heartbeater.cc:503] Master 127.30.194.62:36661 was elected leader, sending a full tablet report...
I20260808 17:01:50.168918 2813 raft_consensus.cc:729] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1249c6fa72ee4ac395cff1be557019a7, State: Initialized, Role: FOLLOWER
I20260808 17:01:50.169051 2811 ts_tablet_manager.cc:1428] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:01:50.169150 2518 heartbeater.cc:503] Master 127.30.194.62:36661 was elected leader, sending a full tablet report...
I20260808 17:01:50.169055 2813 consensus_queue.cc:260] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.169737 2813 ts_tablet_manager.cc:1428] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:01:50.169857 2771 heartbeater.cc:503] Master 127.30.194.62:36661 was elected leader, sending a full tablet report...
W20260808 17:01:50.217500 2519 tablet.cc:2369] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:01:50.329150 2298 tablet.cc:2369] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:01:50.366559 2773 tablet.cc:2369] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:01:50.445523 2823 raft_consensus.cc:482] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:50.445648 2823 raft_consensus.cc:504] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.445966 2823 leader_election.cc:288] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947), 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057)
I20260808 17:01:50.448601 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.446239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:45293 (local address 127.30.194.1:43947)
0808 17:01:50.446292 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.446294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.446306 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:01:50.446338 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.446339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.446356 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.446420 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:50.446571 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.447040 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.447937 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.448295 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.448355 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.448365 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.448503 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.448540 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:01:50.448601 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.446156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:43947 (local address 127.30.194.2:45293)
0808 17:01:50.446211 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.446223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.446272 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.446430 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.446431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.446434 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:50.446506 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.446508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.447113 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.447114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.447844 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.447846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.448366 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.448368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.448442 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.448445 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.448490 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.448525 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":5}
I20260808 17:01:50.448980 2826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.446303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:44057 (local address 127.30.194.2:47585)
0808 17:01:50.446508 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.446517 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.446579 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.446695 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.446696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.446697 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:50.446787 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.446788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.447388 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.447389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.448091 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.448093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.448583 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.448584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.448807 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.448812 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.448870 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.448905 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":124,"threads_started":1}
I20260808 17:01:50.449002 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.446439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:47585 (local address 127.30.194.3:44057)
0808 17:01:50.446480 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.446482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.446576 (+ 94us) server_negotiation.cc:407] Connection header received
0808 17:01:50.446632 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.446633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.446641 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.446688 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:50.446845 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.447320 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.448168 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.448516 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.448575 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.448582 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.448890 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.448933 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:01:50.449010 2202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f515d7055d4a99902c1885210bcf37" candidate_uuid: "754295157b374497891db284c0a93963" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351afd7919474272bc4431d70efb0987" is_pre_election: true
I20260808 17:01:50.449195 2202 raft_consensus.cc:2454] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 754295157b374497891db284c0a93963 in term 0.
I20260808 17:01:50.449352 2676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f515d7055d4a99902c1885210bcf37" candidate_uuid: "754295157b374497891db284c0a93963" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1249c6fa72ee4ac395cff1be557019a7" is_pre_election: true
I20260808 17:01:50.449406 2336 leader_election.cc:302] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351afd7919474272bc4431d70efb0987, 754295157b374497891db284c0a93963; no voters:
I20260808 17:01:50.449496 2676 raft_consensus.cc:2454] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 754295157b374497891db284c0a93963 in term 0.
I20260808 17:01:50.449518 2823 raft_consensus.cc:2781] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:50.449589 2823 raft_consensus.cc:482] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:50.449611 2823 raft_consensus.cc:3037] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:50.450309 2823 raft_consensus.cc:504] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.450410 2823 leader_election.cc:288] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [CANDIDATE]: Term 1 election: Requested vote from peers 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947), 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057)
I20260808 17:01:50.450553 2202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f515d7055d4a99902c1885210bcf37" candidate_uuid: "754295157b374497891db284c0a93963" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351afd7919474272bc4431d70efb0987"
I20260808 17:01:50.450584 2676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f515d7055d4a99902c1885210bcf37" candidate_uuid: "754295157b374497891db284c0a93963" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1249c6fa72ee4ac395cff1be557019a7"
I20260808 17:01:50.450623 2202 raft_consensus.cc:3037] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:50.450661 2676 raft_consensus.cc:3037] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:50.451311 2202 raft_consensus.cc:2454] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 754295157b374497891db284c0a93963 in term 1.
I20260808 17:01:50.451360 2676 raft_consensus.cc:2454] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 754295157b374497891db284c0a93963 in term 1.
I20260808 17:01:50.451491 2336 leader_election.cc:302] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1249c6fa72ee4ac395cff1be557019a7, 754295157b374497891db284c0a93963; no voters:
I20260808 17:01:50.451596 2823 raft_consensus.cc:2781] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:50.451691 2823 raft_consensus.cc:686] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 1 LEADER]: Becoming Leader. State: Replica: 754295157b374497891db284c0a93963, State: Running, Role: LEADER
I20260808 17:01:50.451788 2823 consensus_queue.cc:237] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.452625 1954 catalog_manager.cc:5495] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 reported cstate change: term changed from 0 to 1, leader changed from <none> to 754295157b374497891db284c0a93963 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "754295157b374497891db284c0a93963" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } health_report { overall_health: UNKNOWN } } }
Aug 08 17:01:50 dist-test-slave-2j7r krb5kdc[1880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208509, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:50.527750 2841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.521686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36661 (local address 127.0.0.1:50556)
0808 17:01:50.521837 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.521847 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.521912 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.522023 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.522023 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.522336 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:50.522383 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.522386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.522971 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.522971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.523489 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.523490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.523822 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.523823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.523872 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.524302 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:50.524306 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:50.525633 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:50.526819 (+ 1186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:50.526827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:50.526829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:50.527009 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:50.527169 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:50.527169 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:50.527169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:50.527230 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:50.527421 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:50.527430 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:50.527473 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.527530 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.527640 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260808 17:01:50.527808 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.521767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50556 (local address 127.30.194.62:36661)
0808 17:01:50.521830 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.521832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.521895 (+ 63us) server_negotiation.cc:407] Connection header received
0808 17:01:50.521954 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.521954 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.521963 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.522008 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:50.522444 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.522915 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.523555 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.523740 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.523789 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.525707 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:50.525715 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:50.525715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:50.525721 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:50.526753 (+ 1032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:50.527069 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:50.527071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:50.527071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:50.527118 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:50.527277 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:50.527278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:50.527279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:50.527371 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:50.527424 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.527556 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.527699 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":41}
I20260808 17:01:50.529862 1954 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:50556:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 17:01:50.530071 1954 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:50.531500 2171 tablet_service.cc:1467] Processing CreateTablet for tablet 48bf919b6ce846339c18bb41db897f85 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=755c4506f1eb43b3bfc972672137c039]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:01:50.531512 2614 tablet_service.cc:1467] Processing CreateTablet for tablet 48bf919b6ce846339c18bb41db897f85 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=755c4506f1eb43b3bfc972672137c039]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:01:50.531590 2411 tablet_service.cc:1467] Processing CreateTablet for tablet 48bf919b6ce846339c18bb41db897f85 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=755c4506f1eb43b3bfc972672137c039]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:01:50.531651 2171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48bf919b6ce846339c18bb41db897f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:50.531687 2614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48bf919b6ce846339c18bb41db897f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:50.531713 2411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48bf919b6ce846339c18bb41db897f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:50.532812 2815 tablet_bootstrap.cc:492] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: Bootstrap starting.
I20260808 17:01:50.533131 2813 tablet_bootstrap.cc:492] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: Bootstrap starting.
I20260808 17:01:50.533172 2815 tablet_bootstrap.cc:654] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:50.533674 2813 tablet_bootstrap.cc:654] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:50.533824 2815 tablet_bootstrap.cc:492] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: No bootstrap required, opened a new log
I20260808 17:01:50.533874 2815 ts_tablet_manager.cc:1397] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:50.533973 2811 tablet_bootstrap.cc:492] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: Bootstrap starting.
I20260808 17:01:50.534037 2815 raft_consensus.cc:348] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.534108 2815 raft_consensus.cc:374] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:50.534127 2815 raft_consensus.cc:729] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351afd7919474272bc4431d70efb0987, State: Initialized, Role: FOLLOWER
I20260808 17:01:50.534171 2815 consensus_queue.cc:260] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.534262 2815 ts_tablet_manager.cc:1428] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:50.534335 2811 tablet_bootstrap.cc:654] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:50.534333 2821 tablet_replica.cc:439] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } } }
I20260808 17:01:50.534416 2821 tablet_replica.cc:442] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:50.534675 2813 tablet_bootstrap.cc:492] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: No bootstrap required, opened a new log
I20260808 17:01:50.534739 2813 ts_tablet_manager.cc:1397] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:01:50.534893 2813 raft_consensus.cc:348] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.534971 2813 raft_consensus.cc:374] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:50.534986 2813 raft_consensus.cc:729] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1249c6fa72ee4ac395cff1be557019a7, State: Initialized, Role: FOLLOWER
I20260808 17:01:50.535018 2813 consensus_queue.cc:260] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.535099 2813 ts_tablet_manager.cc:1428] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:50.535216 2811 tablet_bootstrap.cc:492] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: No bootstrap required, opened a new log
I20260808 17:01:50.535277 2811 ts_tablet_manager.cc:1397] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:50.535238 2824 tablet_replica.cc:439] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } } }
I20260808 17:01:50.535324 2824 tablet_replica.cc:442] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:50.535418 2811 raft_consensus.cc:348] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.535485 2811 raft_consensus.cc:374] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:50.535506 2811 raft_consensus.cc:729] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754295157b374497891db284c0a93963, State: Initialized, Role: FOLLOWER
I20260808 17:01:50.535542 2811 consensus_queue.cc:260] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.535629 2811 ts_tablet_manager.cc:1428] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:50.535722 2823 tablet_replica.cc:439] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } } }
I20260808 17:01:50.535804 2823 tablet_replica.cc:442] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:50.634317 2824 raft_consensus.cc:482] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:50.634408 2824 raft_consensus.cc:504] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.634729 2824 leader_election.cc:288] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947), 754295157b374497891db284c0a93963 (127.30.194.2:41279)
I20260808 17:01:50.637383 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.634941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:36705 (local address 127.30.194.1:43947)
0808 17:01:50.634985 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.634987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.635070 (+ 83us) server_negotiation.cc:407] Connection header received
0808 17:01:50.635128 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.635129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.635139 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.635184 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:50.635345 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.635821 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.636741 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.637073 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.637155 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.637164 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.637289 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.637322 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:01:50.637424 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.634967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:42001 (local address 127.30.194.2:41279)
0808 17:01:50.635011 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.635013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.635084 (+ 71us) server_negotiation.cc:407] Connection header received
0808 17:01:50.635137 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.635138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.635147 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.635197 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:50.635374 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.635827 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.636744 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.637123 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.637184 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.637193 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.637325 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.637360 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:01:50.637432 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.634876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:43947 (local address 127.30.194.3:36705)
0808 17:01:50.635011 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.635021 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.635077 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.635197 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.635197 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.635199 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:50.635289 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.635290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.635894 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.635894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.636658 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.636660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.637154 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.637155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.637231 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.637235 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.637278 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.637318 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260808 17:01:50.637444 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.634890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:41279 (local address 127.30.194.3:42001)
0808 17:01:50.635022 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.635032 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.635087 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.635207 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.635207 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.635209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:50.635317 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.635319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.635901 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.635902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.636661 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.636663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.637197 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.637198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.637266 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.637269 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.637314 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.637353 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":82,"thread_start_us":69,"threads_started":1}
I20260808 17:01:50.637663 2202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48bf919b6ce846339c18bb41db897f85" candidate_uuid: "1249c6fa72ee4ac395cff1be557019a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351afd7919474272bc4431d70efb0987" is_pre_election: true
I20260808 17:01:50.637770 2202 raft_consensus.cc:2454] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1249c6fa72ee4ac395cff1be557019a7 in term 0.
I20260808 17:01:50.637761 2444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48bf919b6ce846339c18bb41db897f85" candidate_uuid: "1249c6fa72ee4ac395cff1be557019a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754295157b374497891db284c0a93963" is_pre_election: true
I20260808 17:01:50.637861 2444 raft_consensus.cc:2454] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1249c6fa72ee4ac395cff1be557019a7 in term 0.
I20260808 17:01:50.637939 2555 leader_election.cc:302] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1249c6fa72ee4ac395cff1be557019a7, 351afd7919474272bc4431d70efb0987; no voters:
I20260808 17:01:50.638079 2824 raft_consensus.cc:2781] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:50.638124 2824 raft_consensus.cc:482] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:50.638144 2824 raft_consensus.cc:3037] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:50.638787 2824 raft_consensus.cc:504] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.638906 2824 leader_election.cc:288] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [CANDIDATE]: Term 1 election: Requested vote from peers 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947), 754295157b374497891db284c0a93963 (127.30.194.2:41279)
I20260808 17:01:50.639048 2444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48bf919b6ce846339c18bb41db897f85" candidate_uuid: "1249c6fa72ee4ac395cff1be557019a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754295157b374497891db284c0a93963"
I20260808 17:01:50.639115 2444 raft_consensus.cc:3037] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:50.639108 2202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48bf919b6ce846339c18bb41db897f85" candidate_uuid: "1249c6fa72ee4ac395cff1be557019a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351afd7919474272bc4431d70efb0987"
I20260808 17:01:50.639194 2202 raft_consensus.cc:3037] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:50.639771 2444 raft_consensus.cc:2454] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1249c6fa72ee4ac395cff1be557019a7 in term 1.
I20260808 17:01:50.639779 2202 raft_consensus.cc:2454] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1249c6fa72ee4ac395cff1be557019a7 in term 1.
I20260808 17:01:50.639918 2563 leader_election.cc:302] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1249c6fa72ee4ac395cff1be557019a7, 754295157b374497891db284c0a93963; no voters:
I20260808 17:01:50.640008 2824 raft_consensus.cc:2781] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:50.640120 2824 raft_consensus.cc:686] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 LEADER]: Becoming Leader. State: Replica: 1249c6fa72ee4ac395cff1be557019a7, State: Running, Role: LEADER
I20260808 17:01:50.640200 2824 consensus_queue.cc:237] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.640612 2849 tablet_replica.cc:439] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: TxnStatusTablet state changed. Reason: New leader 1249c6fa72ee4ac395cff1be557019a7. Latest consensus state: current_term: 1 leader_uuid: "1249c6fa72ee4ac395cff1be557019a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } } }
I20260808 17:01:50.640681 2849 tablet_replica.cc:442] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:01:50.640836 2859 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 17:01:50.640933 2859 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 17:01:50.640897 1953 catalog_manager.cc:5495] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3). New cstate: current_term: 1 leader_uuid: "1249c6fa72ee4ac395cff1be557019a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } health_report { overall_health: HEALTHY } } }
I20260808 17:01:50.650331 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.648122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56846 (local address 127.30.194.3:44057)
0808 17:01:50.648173 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.648174 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.648182 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:50.648270 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.648271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.648287 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.648344 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:50.648451 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.648904 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.649643 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.649840 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.649891 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.650059 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.650143 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.650208 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.650249 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.650272 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:01:50.650363 2841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.648039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:44057 (local address 127.0.0.1:56846)
0808 17:01:50.648132 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.648143 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.648201 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.648350 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.648351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.648353 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:01:50.648398 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.648400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.648974 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.648975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.649572 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.649574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.649898 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.649900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.649964 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.649978 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.650197 (+ 219us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.650204 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.650247 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.650298 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 17:01:50.652148 2444 raft_consensus.cc:1261] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Refusing update from remote peer 1249c6fa72ee4ac395cff1be557019a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:50.652182 2202 raft_consensus.cc:1261] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Refusing update from remote peer 1249c6fa72ee4ac395cff1be557019a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:50.652413 2824 consensus_queue.cc:1035] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:50.652531 2849 consensus_queue.cc:1035] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:50.653653 2823 tablet_replica.cc:439] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: TxnStatusTablet state changed. Reason: New leader 1249c6fa72ee4ac395cff1be557019a7. Latest consensus state: current_term: 1 leader_uuid: "1249c6fa72ee4ac395cff1be557019a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } } }
I20260808 17:01:50.653685 2821 tablet_replica.cc:439] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: TxnStatusTablet state changed. Reason: New leader 1249c6fa72ee4ac395cff1be557019a7. Latest consensus state: current_term: 1 leader_uuid: "1249c6fa72ee4ac395cff1be557019a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } } }
I20260808 17:01:50.653738 2823 tablet_replica.cc:442] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:50.653753 2821 tablet_replica.cc:442] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:50.654915 2849 tablet_replica.cc:439] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1249c6fa72ee4ac395cff1be557019a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } } }
I20260808 17:01:50.654992 2849 tablet_replica.cc:442] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:01:50.655036 2849 tablet_replica.cc:439] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1249c6fa72ee4ac395cff1be557019a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } } }
I20260808 17:01:50.655081 2849 tablet_replica.cc:442] T 48bf919b6ce846339c18bb41db897f85 P 1249c6fa72ee4ac395cff1be557019a7: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:01:50.655053 2821 tablet_replica.cc:439] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1249c6fa72ee4ac395cff1be557019a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } } }
I20260808 17:01:50.655126 2821 tablet_replica.cc:442] T 48bf919b6ce846339c18bb41db897f85 P 351afd7919474272bc4431d70efb0987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:50.655707 2823 tablet_replica.cc:439] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1249c6fa72ee4ac395cff1be557019a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } } }
I20260808 17:01:50.655794 2823 tablet_replica.cc:442] T 48bf919b6ce846339c18bb41db897f85 P 754295157b374497891db284c0a93963: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:01:50.656280 2869 mvcc.cc:205] Tried to move back new op lower bound from 7316310059629350912 to 7316310059582799872. Current Snapshot: MvccSnapshot[applied={T|T < 7316310059629350912}]
I20260808 17:01:50.656283 2865 mvcc.cc:205] Tried to move back new op lower bound from 7316310059629350912 to 7316310059582799872. Current Snapshot: MvccSnapshot[applied={T|T < 7316310059629350912 or (T in {7316310059629350912})}]
I20260808 17:01:50.656383 2868 mvcc.cc:205] Tried to move back new op lower bound from 7316310059629350912 to 7316310059582799872. Current Snapshot: MvccSnapshot[applied={T|T < 7316310059629350912}]
I20260808 17:01:50.659551 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.657017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.30.194.2:41279)
0808 17:01:50.657063 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.657065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.657121 (+ 56us) server_negotiation.cc:407] Connection header received
0808 17:01:50.657197 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.657198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.657220 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.657295 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:50.657384 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.658082 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.658790 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.659095 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.659160 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.659263 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.659381 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.659454 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.659468 (+ 14us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.659487 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:01:50.663829 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.661726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:53049 (local address 127.30.194.3:44057)
0808 17:01:50.661765 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.661766 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.661859 (+ 93us) server_negotiation.cc:407] Connection header received
0808 17:01:50.661906 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.661907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.661915 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.661964 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:50.662077 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.662511 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.663220 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.663424 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.663475 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.663634 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.663664 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.663705 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.663754 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.663770 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:01:50.664060 2885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.661660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:44057 (local address 127.30.194.2:53049)
0808 17:01:50.661785 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.661810 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.661864 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.661968 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.661969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.661970 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:01:50.662028 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.662029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.662578 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.662579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.663151 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.663152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.663482 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.663483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.663545 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.663558 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.663712 (+ 154us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.663714 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.663753 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.663788 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":61,"threads_started":1}
I20260808 17:01:50.666162 2676 raft_consensus.cc:1261] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Refusing update from remote peer 754295157b374497891db284c0a93963: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:50.666313 2202 raft_consensus.cc:1261] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Refusing update from remote peer 754295157b374497891db284c0a93963: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:50.666440 2823 consensus_queue.cc:1035] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:50.666544 2828 consensus_queue.cc:1035] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [LEADER]: Connected to new peer: Peer: permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:50.674613 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.672082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:37213 (local address 127.30.194.2:41279)
0808 17:01:50.672129 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.672131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.672254 (+ 123us) server_negotiation.cc:407] Connection header received
0808 17:01:50.672303 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.672303 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.672311 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.672360 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:50.672568 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.672995 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.673749 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.673995 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.674052 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.674440 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.674465 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.674512 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.674553 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.674571 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:01:50.674685 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.672040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:41279 (local address 127.30.194.3:37213)
0808 17:01:50.672188 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.672197 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.672251 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.672432 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.672433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.672435 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:01:50.672487 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.672489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.673062 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.673063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.673678 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.673680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.674148 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.674149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.674297 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.674311 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.674512 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.674514 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.674553 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.674590 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260808 17:01:50.684142 1953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:50.684232 1953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:50.684898 1953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7: Sending DeleteTablet for 3 replicas of tablet 60f515d7055d4a99902c1885210bcf37
I20260808 17:01:50.685210 2171 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f515d7055d4a99902c1885210bcf37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:50 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:57054
I20260808 17:01:50.685259 2411 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f515d7055d4a99902c1885210bcf37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:50 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:57730
I20260808 17:01:50.685491 2906 tablet_replica.cc:330] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: stopping tablet replica
I20260808 17:01:50.685585 2906 raft_consensus.cc:2229] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:01:50.685611 2614 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f515d7055d4a99902c1885210bcf37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:50 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:56840
I20260808 17:01:50.685684 2906 raft_consensus.cc:2258] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:50.686273 2906 ts_tablet_manager.cc:1905] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:50.686269 1953 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:01:50.686405 1953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:50.686862 2908 tablet_replica.cc:330] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: stopping tablet replica
I20260808 17:01:50.687005 2908 raft_consensus.cc:2229] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:50.687119 2908 raft_consensus.cc:2258] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:50.687460 2908 ts_tablet_manager.cc:1905] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:50.687832 2411 tablet_service.cc:1467] Processing CreateTablet for tablet f79afed6d404482094208dfdf9ff012a (DEFAULT_TABLE table=test-table [id=17a5aa8693794dbeb172b7bf33df89be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:50.687914 2171 tablet_service.cc:1467] Processing CreateTablet for tablet f79afed6d404482094208dfdf9ff012a (DEFAULT_TABLE table=test-table [id=17a5aa8693794dbeb172b7bf33df89be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:50.687947 2411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f79afed6d404482094208dfdf9ff012a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:50.687999 2171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f79afed6d404482094208dfdf9ff012a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:50.689232 2811 tablet_bootstrap.cc:492] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963: Bootstrap starting.
I20260808 17:01:50.689272 2614 tablet_service.cc:1467] Processing CreateTablet for tablet f79afed6d404482094208dfdf9ff012a (DEFAULT_TABLE table=test-table [id=17a5aa8693794dbeb172b7bf33df89be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:50.689368 2614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f79afed6d404482094208dfdf9ff012a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:50.689648 2811 tablet_bootstrap.cc:654] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:50.689679 2906 ts_tablet_manager.cc:1918] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:01:50.689750 2906 log.cc:1196] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/60f515d7055d4a99902c1885210bcf37
I20260808 17:01:50.690012 2906 ts_tablet_manager.cc:1939] T 60f515d7055d4a99902c1885210bcf37 P 754295157b374497891db284c0a93963: Deleting consensus metadata
I20260808 17:01:50.690471 1933 catalog_manager.cc:4841] TS 754295157b374497891db284c0a93963 (127.30.194.2:41279): tablet 60f515d7055d4a99902c1885210bcf37 (table test-table [id=7d5e50f4a16f45119d7795611353304a]) successfully deleted
I20260808 17:01:50.690650 2815 tablet_bootstrap.cc:492] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987: Bootstrap starting.
I20260808 17:01:50.691289 2815 tablet_bootstrap.cc:654] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:50.691696 2813 tablet_bootstrap.cc:492] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7: Bootstrap starting.
I20260808 17:01:50.692102 2813 tablet_bootstrap.cc:654] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:50.692857 2813 tablet_bootstrap.cc:492] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7: No bootstrap required, opened a new log
I20260808 17:01:50.692909 2813 ts_tablet_manager.cc:1397] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:50.693038 2813 raft_consensus.cc:348] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.693127 2813 raft_consensus.cc:374] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:50.693150 2813 raft_consensus.cc:729] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1249c6fa72ee4ac395cff1be557019a7, State: Initialized, Role: FOLLOWER
I20260808 17:01:50.693179 2905 tablet_replica.cc:330] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: stopping tablet replica
I20260808 17:01:50.693195 2813 consensus_queue.cc:260] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.693280 2813 ts_tablet_manager.cc:1428] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:50.693282 2905 raft_consensus.cc:2229] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:50.693336 2905 raft_consensus.cc:2258] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:50.693503 2815 tablet_bootstrap.cc:492] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987: No bootstrap required, opened a new log
I20260808 17:01:50.693543 2815 ts_tablet_manager.cc:1397] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 17:01:50.693745 2905 ts_tablet_manager.cc:1905] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:50.693701 2815 raft_consensus.cc:348] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.693820 2815 raft_consensus.cc:374] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:50.693876 2815 raft_consensus.cc:729] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351afd7919474272bc4431d70efb0987, State: Initialized, Role: FOLLOWER
I20260808 17:01:50.693956 2815 consensus_queue.cc:260] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.694061 2815 ts_tablet_manager.cc:1428] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 17:01:50.695256 2905 ts_tablet_manager.cc:1918] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:01:50.695256 2908 ts_tablet_manager.cc:1918] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:01:50.695338 2908 log.cc:1196] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/60f515d7055d4a99902c1885210bcf37
I20260808 17:01:50.695338 2905 log.cc:1196] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/60f515d7055d4a99902c1885210bcf37
I20260808 17:01:50.695604 2905 ts_tablet_manager.cc:1939] T 60f515d7055d4a99902c1885210bcf37 P 351afd7919474272bc4431d70efb0987: Deleting consensus metadata
I20260808 17:01:50.695604 2908 ts_tablet_manager.cc:1939] T 60f515d7055d4a99902c1885210bcf37 P 1249c6fa72ee4ac395cff1be557019a7: Deleting consensus metadata
I20260808 17:01:50.696003 1925 catalog_manager.cc:4841] TS 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947): tablet 60f515d7055d4a99902c1885210bcf37 (table test-table [id=7d5e50f4a16f45119d7795611353304a]) successfully deleted
I20260808 17:01:50.696098 1925 catalog_manager.cc:4841] TS 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057): tablet 60f515d7055d4a99902c1885210bcf37 (table test-table [id=7d5e50f4a16f45119d7795611353304a]) successfully deleted
I20260808 17:01:50.697501 2811 tablet_bootstrap.cc:492] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963: No bootstrap required, opened a new log
I20260808 17:01:50.697552 2811 ts_tablet_manager.cc:1397] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260808 17:01:50.697685 2811 raft_consensus.cc:348] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.697748 2811 raft_consensus.cc:374] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:50.697767 2811 raft_consensus.cc:729] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754295157b374497891db284c0a93963, State: Initialized, Role: FOLLOWER
I20260808 17:01:50.697818 2811 consensus_queue.cc:260] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.697906 2811 ts_tablet_manager.cc:1428] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:50.714389 2821 raft_consensus.cc:482] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:50.714478 2821 raft_consensus.cc:504] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.714798 2821 leader_election.cc:288] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 754295157b374497891db284c0a93963 (127.30.194.2:41279), 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057)
I20260808 17:01:50.718647 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.715207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:47861 (local address 127.30.194.2:41279)
0808 17:01:50.715303 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.715305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.715315 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:01:50.715342 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.715342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.715353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.715421 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:50.715600 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.716340 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.717784 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.718284 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.718334 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.718343 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.718550 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.718581 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 17:01:50.718997 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.715254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:42783 (local address 127.30.194.3:44057)
0808 17:01:50.715300 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.715302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.715562 (+ 260us) server_negotiation.cc:407] Connection header received
0808 17:01:50.715667 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.715668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.715677 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.715708 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:50.715935 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.716645 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.718070 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.718581 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.718634 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.718643 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.718894 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.718928 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:01:50.719246 2916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.715006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:44057 (local address 127.30.194.1:42783)
0808 17:01:50.715501 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.715512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.715590 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.715770 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.715771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.715772 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:50.715853 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.715855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.716756 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.716756 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.717955 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.717958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.718713 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.718714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.718804 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.718808 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.719111 (+ 303us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.719153 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":12,"thread_start_us":87,"threads_started":1}
I20260808 17:01:50.719484 2676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79afed6d404482094208dfdf9ff012a" candidate_uuid: "351afd7919474272bc4431d70efb0987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1249c6fa72ee4ac395cff1be557019a7" is_pre_election: true
I20260808 17:01:50.719578 2676 raft_consensus.cc:2454] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 351afd7919474272bc4431d70efb0987 in term 0.
I20260808 17:01:50.719738 2915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.714943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:41279 (local address 127.30.194.1:47861)
0808 17:01:50.715148 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.715159 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.715209 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.715433 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.715435 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.715437 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:50.715514 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.715515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.716432 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.716432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.717685 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.717688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.718402 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.718403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.718475 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.718479 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.718904 (+ 425us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.719658 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":65,"threads_started":1}
I20260808 17:01:50.719759 2095 leader_election.cc:302] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1249c6fa72ee4ac395cff1be557019a7, 351afd7919474272bc4431d70efb0987; no voters:
I20260808 17:01:50.719980 2821 raft_consensus.cc:2781] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:50.720016 2821 raft_consensus.cc:482] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:50.720037 2821 raft_consensus.cc:3037] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:50.720093 2444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79afed6d404482094208dfdf9ff012a" candidate_uuid: "351afd7919474272bc4431d70efb0987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754295157b374497891db284c0a93963" is_pre_election: true
I20260808 17:01:50.720165 2444 raft_consensus.cc:2454] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 351afd7919474272bc4431d70efb0987 in term 0.
I20260808 17:01:50.720908 2821 raft_consensus.cc:504] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.721218 2444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79afed6d404482094208dfdf9ff012a" candidate_uuid: "351afd7919474272bc4431d70efb0987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754295157b374497891db284c0a93963"
I20260808 17:01:50.721287 2444 raft_consensus.cc:3037] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:50.721282 2676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79afed6d404482094208dfdf9ff012a" candidate_uuid: "351afd7919474272bc4431d70efb0987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1249c6fa72ee4ac395cff1be557019a7"
I20260808 17:01:50.721354 2676 raft_consensus.cc:3037] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:50.722069 2676 raft_consensus.cc:2454] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 351afd7919474272bc4431d70efb0987 in term 1.
I20260808 17:01:50.721055 2821 leader_election.cc:288] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [CANDIDATE]: Term 1 election: Requested vote from peers 754295157b374497891db284c0a93963 (127.30.194.2:41279), 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057)
I20260808 17:01:50.722249 2095 leader_election.cc:302] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1249c6fa72ee4ac395cff1be557019a7, 351afd7919474272bc4431d70efb0987; no voters:
I20260808 17:01:50.722360 2821 raft_consensus.cc:2781] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:50.722442 2444 raft_consensus.cc:2454] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 351afd7919474272bc4431d70efb0987 in term 1.
I20260808 17:01:50.722497 2821 raft_consensus.cc:686] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 1 LEADER]: Becoming Leader. State: Replica: 351afd7919474272bc4431d70efb0987, State: Running, Role: LEADER
I20260808 17:01:50.722563 2821 consensus_queue.cc:237] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:50.723141 1953 catalog_manager.cc:5495] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 reported cstate change: term changed from 0 to 1, leader changed from <none> to 351afd7919474272bc4431d70efb0987 (127.30.194.1). New cstate: current_term: 1 leader_uuid: "351afd7919474272bc4431d70efb0987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } health_report { overall_health: UNKNOWN } } }
I20260808 17:01:50.734184 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.731211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57062 (local address 127.30.194.1:43947)
0808 17:01:50.731246 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.731249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.731274 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:01:50.731304 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.731305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.731324 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.731388 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:50.731527 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.732237 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.733280 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.733575 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.733644 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.733830 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.733949 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.734020 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.734098 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.734119 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:01:50.744160 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.736781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:34783 (local address 127.30.194.3:44057)
0808 17:01:50.736821 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.736823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.741306 (+ 4483us) server_negotiation.cc:407] Connection header received
0808 17:01:50.741341 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.741342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.741354 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.741427 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:50.741560 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.742241 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.743310 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.743600 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.743678 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.743897 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.743933 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.743991 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.744074 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.744094 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:01:50.744190 2930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.736701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:44057 (local address 127.30.194.1:34783)
0808 17:01:50.741227 (+ 4526us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.741238 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.741298 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.741429 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.741430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.741431 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:01:50.741498 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.741499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.742324 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.742325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.743220 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.743223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.743679 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.743680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.743747 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.743761 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.744024 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.744026 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.744065 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.744106 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4485,"thread_start_us":77,"threads_started":1}
I20260808 17:01:50.746760 2444 raft_consensus.cc:1261] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Refusing update from remote peer 351afd7919474272bc4431d70efb0987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:50.746774 2676 raft_consensus.cc:1261] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Refusing update from remote peer 351afd7919474272bc4431d70efb0987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:50.747004 2821 consensus_queue.cc:1035] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:50.747984 2821 consensus_queue.cc:1035] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:50.755292 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.752412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:43947 (local address 127.30.194.3:58047)
0808 17:01:50.752524 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:50.752534 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:50.752584 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:50.752753 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:50.752754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:50.752755 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:01:50.752799 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.752801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.753646 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.753647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.754465 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:50.754468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.754938 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.754939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:50.754993 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.755000 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.755151 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.755153 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:01:50.755186 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 17:01:50.755225 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 17:01:50.755669 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:50.752491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:58047 (local address 127.30.194.1:43947)
0808 17:01:50.752524 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:50.752525 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:50.752571 (+ 46us) server_negotiation.cc:407] Connection header received
0808 17:01:50.752641 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:50.752642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:50.752650 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:50.752698 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:50.752867 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.753559 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.754558 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:50.754841 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:50.754904 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:50.755069 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:50.755096 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:50.755131 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:50.755591 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 17:01:50.755612 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 17:01:50.756037 2869 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 17:01:52.284363 1953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:52.284461 1953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:52.285192 1953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7: Sending DeleteTablet for 3 replicas of tablet f79afed6d404482094208dfdf9ff012a
I20260808 17:01:52.285471 2614 tablet_service.cc:1514] Processing DeleteTablet for tablet f79afed6d404482094208dfdf9ff012a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:52 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:56840
I20260808 17:01:52.285470 2171 tablet_service.cc:1514] Processing DeleteTablet for tablet f79afed6d404482094208dfdf9ff012a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:52 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:57054
I20260808 17:01:52.285497 2411 tablet_service.cc:1514] Processing DeleteTablet for tablet f79afed6d404482094208dfdf9ff012a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:52 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:57730
I20260808 17:01:52.285691 2975 tablet_replica.cc:330] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963: stopping tablet replica
I20260808 17:01:52.285712 2973 tablet_replica.cc:330] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987: stopping tablet replica
I20260808 17:01:52.285763 2973 raft_consensus.cc:2229] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:01:52.285759 2975 raft_consensus.cc:2229] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:52.285847 2975 raft_consensus.cc:2258] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:52.285857 2973 raft_consensus.cc:2258] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:52.286085 2975 ts_tablet_manager.cc:1905] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:52.286213 2973 ts_tablet_manager.cc:1905] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:52.287814 2974 tablet_replica.cc:330] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7: stopping tablet replica
I20260808 17:01:52.287874 2974 raft_consensus.cc:2229] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:52.287901 2974 raft_consensus.cc:2258] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:52.288012 2973 ts_tablet_manager.cc:1918] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 17:01:52.288028 2975 ts_tablet_manager.cc:1918] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 17:01:52.288069 2973 log.cc:1196] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/f79afed6d404482094208dfdf9ff012a
I20260808 17:01:52.288070 2975 log.cc:1196] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/f79afed6d404482094208dfdf9ff012a
I20260808 17:01:52.288317 2973 ts_tablet_manager.cc:1939] T f79afed6d404482094208dfdf9ff012a P 351afd7919474272bc4431d70efb0987: Deleting consensus metadata
I20260808 17:01:52.288406 2975 ts_tablet_manager.cc:1939] T f79afed6d404482094208dfdf9ff012a P 754295157b374497891db284c0a93963: Deleting consensus metadata
I20260808 17:01:52.288597 1925 catalog_manager.cc:4841] TS 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947): tablet f79afed6d404482094208dfdf9ff012a (table test-table [id=17a5aa8693794dbeb172b7bf33df89be]) successfully deleted
I20260808 17:01:52.288759 1933 catalog_manager.cc:4841] TS 754295157b374497891db284c0a93963 (127.30.194.2:41279): tablet f79afed6d404482094208dfdf9ff012a (table test-table [id=17a5aa8693794dbeb172b7bf33df89be]) successfully deleted
I20260808 17:01:52.288905 2974 ts_tablet_manager.cc:1905] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:52.290167 2974 ts_tablet_manager.cc:1918] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 17:01:52.290227 2974 log.cc:1196] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/f79afed6d404482094208dfdf9ff012a
I20260808 17:01:52.290407 2974 ts_tablet_manager.cc:1939] T f79afed6d404482094208dfdf9ff012a P 1249c6fa72ee4ac395cff1be557019a7: Deleting consensus metadata
I20260808 17:01:52.290699 1925 catalog_manager.cc:4841] TS 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057): tablet f79afed6d404482094208dfdf9ff012a (table test-table [id=17a5aa8693794dbeb172b7bf33df89be]) successfully deleted
Aug 08 17:01:52 dist-test-slave-2j7r krb5kdc[1880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208512, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 17:01:52 dist-test-slave-2j7r krb5kdc[1880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208512, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:52.309222 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:52.302684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50558 (local address 127.30.194.62:36661)
0808 17:01:52.302826 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:52.302828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:52.302836 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:52.302887 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:52.302887 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:52.302897 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:52.302952 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:52.303471 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:52.303969 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:52.304706 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:52.304918 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:52.304975 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:52.306970 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:52.306972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:52.306972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:52.306978 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:52.308258 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:52.308581 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:52.308581 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:52.308582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:52.308633 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:52.308769 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:52.308770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:52.308770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:52.308863 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:52.308932 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:52.309006 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 17:01:52.309066 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":96,"threads_started":1}
I20260808 17:01:52.310600 1953 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:01:52.310755 1953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:52.311896 2411 tablet_service.cc:1467] Processing CreateTablet for tablet 2f77c332dbb345f598ab30bd0f0babb7 (DEFAULT_TABLE table=test-table [id=b3d71fea048e4314beaf669ec06dc88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:52.311939 2614 tablet_service.cc:1467] Processing CreateTablet for tablet 2f77c332dbb345f598ab30bd0f0babb7 (DEFAULT_TABLE table=test-table [id=b3d71fea048e4314beaf669ec06dc88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:52.311951 2171 tablet_service.cc:1467] Processing CreateTablet for tablet 2f77c332dbb345f598ab30bd0f0babb7 (DEFAULT_TABLE table=test-table [id=b3d71fea048e4314beaf669ec06dc88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:52.312029 2411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f77c332dbb345f598ab30bd0f0babb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:52.312044 2171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f77c332dbb345f598ab30bd0f0babb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:52.312058 2614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f77c332dbb345f598ab30bd0f0babb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:52.312971 2992 tablet_bootstrap.cc:492] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7: Bootstrap starting.
I20260808 17:01:52.313369 2992 tablet_bootstrap.cc:654] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:52.313532 2996 tablet_bootstrap.cc:492] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987: Bootstrap starting.
I20260808 17:01:52.313849 2996 tablet_bootstrap.cc:654] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:52.314016 2992 tablet_bootstrap.cc:492] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7: No bootstrap required, opened a new log
I20260808 17:01:52.314059 2992 ts_tablet_manager.cc:1397] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:52.314141 2998 tablet_bootstrap.cc:492] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963: Bootstrap starting.
I20260808 17:01:52.314182 2992 raft_consensus.cc:348] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.314283 2992 raft_consensus.cc:374] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:52.314306 2992 raft_consensus.cc:729] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1249c6fa72ee4ac395cff1be557019a7, State: Initialized, Role: FOLLOWER
I20260808 17:01:52.314358 2992 consensus_queue.cc:260] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.314450 2992 ts_tablet_manager.cc:1428] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:52.314497 2998 tablet_bootstrap.cc:654] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:52.314863 2996 tablet_bootstrap.cc:492] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987: No bootstrap required, opened a new log
I20260808 17:01:52.314936 2996 ts_tablet_manager.cc:1397] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:52.315129 2998 tablet_bootstrap.cc:492] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963: No bootstrap required, opened a new log
I20260808 17:01:52.315112 2996 raft_consensus.cc:348] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.315179 2996 raft_consensus.cc:374] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:52.315204 2996 raft_consensus.cc:729] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351afd7919474272bc4431d70efb0987, State: Initialized, Role: FOLLOWER
I20260808 17:01:52.315259 2996 consensus_queue.cc:260] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.315351 2996 ts_tablet_manager.cc:1428] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:52.315498 2998 ts_tablet_manager.cc:1397] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:52.315629 2998 raft_consensus.cc:348] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.315703 2998 raft_consensus.cc:374] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:52.315732 2998 raft_consensus.cc:729] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754295157b374497891db284c0a93963, State: Initialized, Role: FOLLOWER
I20260808 17:01:52.315796 2998 consensus_queue.cc:260] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.315982 2998 ts_tablet_manager.cc:1428] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:52.509022 2960 raft_consensus.cc:482] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:52.509135 2960 raft_consensus.cc:504] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.509317 2960 leader_election.cc:288] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947), 754295157b374497891db284c0a93963 (127.30.194.2:41279)
I20260808 17:01:52.509509 2202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f77c332dbb345f598ab30bd0f0babb7" candidate_uuid: "1249c6fa72ee4ac395cff1be557019a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351afd7919474272bc4431d70efb0987" is_pre_election: true
I20260808 17:01:52.509516 2444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f77c332dbb345f598ab30bd0f0babb7" candidate_uuid: "1249c6fa72ee4ac395cff1be557019a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754295157b374497891db284c0a93963" is_pre_election: true
I20260808 17:01:52.509598 2202 raft_consensus.cc:2454] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1249c6fa72ee4ac395cff1be557019a7 in term 0.
I20260808 17:01:52.509611 2444 raft_consensus.cc:2454] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1249c6fa72ee4ac395cff1be557019a7 in term 0.
I20260808 17:01:52.509755 2555 leader_election.cc:302] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1249c6fa72ee4ac395cff1be557019a7, 351afd7919474272bc4431d70efb0987; no voters:
I20260808 17:01:52.509876 2960 raft_consensus.cc:2781] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:52.509917 2960 raft_consensus.cc:482] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:52.509930 2960 raft_consensus.cc:3037] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:52.510607 2960 raft_consensus.cc:504] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.510733 2960 leader_election.cc:288] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [CANDIDATE]: Term 1 election: Requested vote from peers 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947), 754295157b374497891db284c0a93963 (127.30.194.2:41279)
I20260808 17:01:52.510862 2202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f77c332dbb345f598ab30bd0f0babb7" candidate_uuid: "1249c6fa72ee4ac395cff1be557019a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351afd7919474272bc4431d70efb0987"
I20260808 17:01:52.510865 2444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f77c332dbb345f598ab30bd0f0babb7" candidate_uuid: "1249c6fa72ee4ac395cff1be557019a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754295157b374497891db284c0a93963"
I20260808 17:01:52.510926 2444 raft_consensus.cc:3037] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:52.510926 2202 raft_consensus.cc:3037] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:52.511513 2444 raft_consensus.cc:2454] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1249c6fa72ee4ac395cff1be557019a7 in term 1.
I20260808 17:01:52.511514 2202 raft_consensus.cc:2454] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1249c6fa72ee4ac395cff1be557019a7 in term 1.
I20260808 17:01:52.511680 2563 leader_election.cc:302] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1249c6fa72ee4ac395cff1be557019a7, 754295157b374497891db284c0a93963; no voters:
I20260808 17:01:52.511781 2960 raft_consensus.cc:2781] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:52.511842 2960 raft_consensus.cc:686] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 LEADER]: Becoming Leader. State: Replica: 1249c6fa72ee4ac395cff1be557019a7, State: Running, Role: LEADER
I20260808 17:01:52.511888 2960 consensus_queue.cc:237] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.512509 1953 catalog_manager.cc:5495] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3). New cstate: current_term: 1 leader_uuid: "1249c6fa72ee4ac395cff1be557019a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } health_report { overall_health: HEALTHY } } }
I20260808 17:01:52.542243 3014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:52.539930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56858 (local address 127.30.194.3:44057)
0808 17:01:52.540088 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:52.540089 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:52.540096 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:01:52.540114 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:52.540114 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:52.540123 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:52.540176 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:52.540329 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:52.540818 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:52.541581 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:52.541786 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:52.541842 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:52.541993 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:52.542035 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:52.542077 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:52.542135 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 17:01:52.542156 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260808 17:01:52.546258 3014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:52.544317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:57431 (local address 127.30.194.3:44057)
0808 17:01:52.544349 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:52.544351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:52.544358 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:01:52.544412 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:52.544413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:52.544422 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:52.544464 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:52.544549 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:52.544979 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:52.545696 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:52.545874 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:52.545925 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:52.546049 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:52.546075 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:52.546113 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:52.546160 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 17:01:52.546179 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:01:52.546322 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:52.544176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:44057 (local address 127.30.194.3:57431)
0808 17:01:52.544309 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:52.544319 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:52.544370 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:52.544466 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:52.544467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:52.544468 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:01:52.544508 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:52.544509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:52.545046 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:52.545047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:52.545628 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:52.545629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:52.545930 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:52.545931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:52.545990 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:52.545994 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:01:52.546120 (+ 126us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:01:52.546121 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 17:01:52.546160 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 17:01:52.546195 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260808 17:01:52.548072 2444 raft_consensus.cc:1261] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Refusing update from remote peer 1249c6fa72ee4ac395cff1be557019a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:52.548204 2202 raft_consensus.cc:1261] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Refusing update from remote peer 1249c6fa72ee4ac395cff1be557019a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:52.548321 2960 consensus_queue.cc:1035] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:52.548429 2960 consensus_queue.cc:1035] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:52.558534 1953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:52.558607 1953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:52.559145 1953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7: Sending DeleteTablet for 3 replicas of tablet 2f77c332dbb345f598ab30bd0f0babb7
I20260808 17:01:52.559391 2171 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f77c332dbb345f598ab30bd0f0babb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:52 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:57054
I20260808 17:01:52.559386 2411 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f77c332dbb345f598ab30bd0f0babb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:52 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:57730
I20260808 17:01:52.559478 2973 tablet_replica.cc:330] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987: stopping tablet replica
I20260808 17:01:52.559501 2975 tablet_replica.cc:330] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963: stopping tablet replica
I20260808 17:01:52.559530 2973 raft_consensus.cc:2229] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:52.559566 2975 raft_consensus.cc:2229] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:52.559578 2973 raft_consensus.cc:2258] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:52.559608 2975 raft_consensus.cc:2258] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:52.559856 2614 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f77c332dbb345f598ab30bd0f0babb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:52 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:56840
I20260808 17:01:52.559803 1953 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 17:01:52.559890 2975 ts_tablet_manager.cc:1905] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:52.559928 2974 tablet_replica.cc:330] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7: stopping tablet replica
W20260808 17:01:52.559935 1953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:52.559971 2974 raft_consensus.cc:2229] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:01:52.560060 2974 raft_consensus.cc:2258] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:52.560360 2973 ts_tablet_manager.cc:1905] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:52.560493 2974 ts_tablet_manager.cc:1905] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:52.561543 2975 ts_tablet_manager.cc:1918] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:01:52.561599 2975 log.cc:1196] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/2f77c332dbb345f598ab30bd0f0babb7
I20260808 17:01:52.561854 2975 ts_tablet_manager.cc:1939] T 2f77c332dbb345f598ab30bd0f0babb7 P 754295157b374497891db284c0a93963: Deleting consensus metadata
I20260808 17:01:52.561856 2973 ts_tablet_manager.cc:1918] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:01:52.561856 2974 ts_tablet_manager.cc:1918] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:01:52.561904 2974 log.cc:1196] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/2f77c332dbb345f598ab30bd0f0babb7
I20260808 17:01:52.561903 2973 log.cc:1196] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/2f77c332dbb345f598ab30bd0f0babb7
I20260808 17:01:52.562095 2973 ts_tablet_manager.cc:1939] T 2f77c332dbb345f598ab30bd0f0babb7 P 351afd7919474272bc4431d70efb0987: Deleting consensus metadata
I20260808 17:01:52.562095 2974 ts_tablet_manager.cc:1939] T 2f77c332dbb345f598ab30bd0f0babb7 P 1249c6fa72ee4ac395cff1be557019a7: Deleting consensus metadata
I20260808 17:01:52.562176 1933 catalog_manager.cc:4841] TS 754295157b374497891db284c0a93963 (127.30.194.2:41279): tablet 2f77c332dbb345f598ab30bd0f0babb7 (table test-table [id=b3d71fea048e4314beaf669ec06dc88b]) successfully deleted
I20260808 17:01:52.562520 1925 catalog_manager.cc:4841] TS 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947): tablet 2f77c332dbb345f598ab30bd0f0babb7 (table test-table [id=b3d71fea048e4314beaf669ec06dc88b]) successfully deleted
I20260808 17:01:52.562840 2171 tablet_service.cc:1467] Processing CreateTablet for tablet c23f28e1595546559d274451d0394609 (DEFAULT_TABLE table=test-table [id=927b7170902f4e4e835a23b569cac402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:52.562887 2411 tablet_service.cc:1467] Processing CreateTablet for tablet c23f28e1595546559d274451d0394609 (DEFAULT_TABLE table=test-table [id=927b7170902f4e4e835a23b569cac402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:52.562948 2171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c23f28e1595546559d274451d0394609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:52.562983 2411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c23f28e1595546559d274451d0394609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:52.564065 2998 tablet_bootstrap.cc:492] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963: Bootstrap starting.
I20260808 17:01:52.564065 2614 tablet_service.cc:1467] Processing CreateTablet for tablet c23f28e1595546559d274451d0394609 (DEFAULT_TABLE table=test-table [id=927b7170902f4e4e835a23b569cac402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:52.564162 2614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c23f28e1595546559d274451d0394609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:52.564406 2998 tablet_bootstrap.cc:654] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:52.564601 1925 catalog_manager.cc:4841] TS 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057): tablet 2f77c332dbb345f598ab30bd0f0babb7 (table test-table [id=b3d71fea048e4314beaf669ec06dc88b]) successfully deleted
I20260808 17:01:52.565160 2992 tablet_bootstrap.cc:492] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7: Bootstrap starting.
I20260808 17:01:52.565183 2996 tablet_bootstrap.cc:492] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987: Bootstrap starting.
I20260808 17:01:52.565485 2992 tablet_bootstrap.cc:654] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:52.565485 2996 tablet_bootstrap.cc:654] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:52.566056 2996 tablet_bootstrap.cc:492] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987: No bootstrap required, opened a new log
I20260808 17:01:52.566108 2996 ts_tablet_manager.cc:1397] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:52.566205 2996 raft_consensus.cc:348] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.566258 2996 raft_consensus.cc:374] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:52.566269 2996 raft_consensus.cc:729] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351afd7919474272bc4431d70efb0987, State: Initialized, Role: FOLLOWER
I20260808 17:01:52.566303 2992 tablet_bootstrap.cc:492] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7: No bootstrap required, opened a new log
I20260808 17:01:52.566303 2996 consensus_queue.cc:260] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.566350 2992 ts_tablet_manager.cc:1397] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:52.566382 2996 ts_tablet_manager.cc:1428] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:52.566444 2992 raft_consensus.cc:348] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.566504 2992 raft_consensus.cc:374] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:52.566524 2992 raft_consensus.cc:729] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1249c6fa72ee4ac395cff1be557019a7, State: Initialized, Role: FOLLOWER
I20260808 17:01:52.566573 2992 consensus_queue.cc:260] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.566664 2992 ts_tablet_manager.cc:1428] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:52.566870 2998 tablet_bootstrap.cc:492] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963: No bootstrap required, opened a new log
I20260808 17:01:52.566913 2998 ts_tablet_manager.cc:1397] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 17:01:52.567014 2998 raft_consensus.cc:348] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.567073 2998 raft_consensus.cc:374] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:52.567093 2998 raft_consensus.cc:729] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754295157b374497891db284c0a93963, State: Initialized, Role: FOLLOWER
I20260808 17:01:52.567142 2998 consensus_queue.cc:260] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.567261 2998 ts_tablet_manager.cc:1428] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:52.621299 2917 raft_consensus.cc:482] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:52.621408 2917 raft_consensus.cc:504] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.621622 2917 leader_election.cc:288] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 754295157b374497891db284c0a93963 (127.30.194.2:41279), 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057)
I20260808 17:01:52.621786 2444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23f28e1595546559d274451d0394609" candidate_uuid: "351afd7919474272bc4431d70efb0987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754295157b374497891db284c0a93963" is_pre_election: true
I20260808 17:01:52.621804 2676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23f28e1595546559d274451d0394609" candidate_uuid: "351afd7919474272bc4431d70efb0987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1249c6fa72ee4ac395cff1be557019a7" is_pre_election: true
I20260808 17:01:52.621888 2444 raft_consensus.cc:2454] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 351afd7919474272bc4431d70efb0987 in term 0.
I20260808 17:01:52.621898 2676 raft_consensus.cc:2454] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 351afd7919474272bc4431d70efb0987 in term 0.
I20260808 17:01:52.622047 2099 leader_election.cc:302] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351afd7919474272bc4431d70efb0987, 754295157b374497891db284c0a93963; no voters:
I20260808 17:01:52.622156 2917 raft_consensus.cc:2781] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:52.622234 2917 raft_consensus.cc:482] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:52.622270 2917 raft_consensus.cc:3037] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:52.623225 2917 raft_consensus.cc:504] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.623387 2917 leader_election.cc:288] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [CANDIDATE]: Term 1 election: Requested vote from peers 754295157b374497891db284c0a93963 (127.30.194.2:41279), 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057)
I20260808 17:01:52.623543 2444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23f28e1595546559d274451d0394609" candidate_uuid: "351afd7919474272bc4431d70efb0987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754295157b374497891db284c0a93963"
I20260808 17:01:52.623587 2676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23f28e1595546559d274451d0394609" candidate_uuid: "351afd7919474272bc4431d70efb0987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1249c6fa72ee4ac395cff1be557019a7"
I20260808 17:01:52.623636 2444 raft_consensus.cc:3037] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:52.623648 2676 raft_consensus.cc:3037] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:52.624330 2676 raft_consensus.cc:2454] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 351afd7919474272bc4431d70efb0987 in term 1.
I20260808 17:01:52.624339 2444 raft_consensus.cc:2454] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 351afd7919474272bc4431d70efb0987 in term 1.
I20260808 17:01:52.624501 2095 leader_election.cc:302] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1249c6fa72ee4ac395cff1be557019a7, 351afd7919474272bc4431d70efb0987; no voters:
I20260808 17:01:52.624619 2917 raft_consensus.cc:2781] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:52.624692 2917 raft_consensus.cc:686] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 1 LEADER]: Becoming Leader. State: Replica: 351afd7919474272bc4431d70efb0987, State: Running, Role: LEADER
I20260808 17:01:52.624755 2917 consensus_queue.cc:237] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } }
I20260808 17:01:52.625451 1954 catalog_manager.cc:5495] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 reported cstate change: term changed from 0 to 1, leader changed from <none> to 351afd7919474272bc4431d70efb0987 (127.30.194.1). New cstate: current_term: 1 leader_uuid: "351afd7919474272bc4431d70efb0987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351afd7919474272bc4431d70efb0987" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 } health_report { overall_health: UNKNOWN } } }
I20260808 17:01:52.642643 3032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:52.640432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57064 (local address 127.30.194.1:43947)
0808 17:01:52.640583 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:52.640584 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:52.640591 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:01:52.640608 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:52.640609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:52.640618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:52.640677 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:52.640811 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:52.641357 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:52.642051 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:52.642238 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:52.642298 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:52.642424 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:52.642454 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:52.642494 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:52.642548 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 17:01:52.642569 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260808 17:01:52.644819 2676 raft_consensus.cc:1261] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Refusing update from remote peer 351afd7919474272bc4431d70efb0987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:52.644830 2444 raft_consensus.cc:1261] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Refusing update from remote peer 351afd7919474272bc4431d70efb0987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:52.645026 2917 consensus_queue.cc:1035] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1249c6fa72ee4ac395cff1be557019a7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:52.645160 2917 consensus_queue.cc:1035] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754295157b374497891db284c0a93963" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 17:01:52.649462 2944 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 17:01:54.166340 1953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:54.166440 1953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:54.167196 1953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 482478e7fe81458b8426da286003a3f7: Sending DeleteTablet for 3 replicas of tablet c23f28e1595546559d274451d0394609
I20260808 17:01:54.167431 2171 tablet_service.cc:1514] Processing DeleteTablet for tablet c23f28e1595546559d274451d0394609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:54 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:57054
I20260808 17:01:54.167448 2614 tablet_service.cc:1514] Processing DeleteTablet for tablet c23f28e1595546559d274451d0394609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:54 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:56840
I20260808 17:01:54.167454 2411 tablet_service.cc:1514] Processing DeleteTablet for tablet c23f28e1595546559d274451d0394609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:54 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:57730
I20260808 17:01:54.167629 3073 tablet_replica.cc:330] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987: stopping tablet replica
I20260808 17:01:54.167680 3073 raft_consensus.cc:2229] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:01:54.167748 3073 raft_consensus.cc:2258] T c23f28e1595546559d274451d0394609 P 351afd7919474272bc4431d70efb0987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:54.167745 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 2071
I20260808 17:01:54.167842 3075 tablet_replica.cc:330] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7: stopping tablet replica
I20260808 17:01:54.173339 3075 raft_consensus.cc:2229] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:54.173372 3075 raft_consensus.cc:2258] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:54.175192 3075 ts_tablet_manager.cc:1905] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:54.176707 3075 ts_tablet_manager.cc:1918] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 17:01:54.176769 3075 log.cc:1196] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/c23f28e1595546559d274451d0394609
I20260808 17:01:54.176970 3075 ts_tablet_manager.cc:1939] T c23f28e1595546559d274451d0394609 P 1249c6fa72ee4ac395cff1be557019a7: Deleting consensus metadata
I20260808 17:01:54.177314 1925 catalog_manager.cc:4841] TS 1249c6fa72ee4ac395cff1be557019a7 (127.30.194.3:44057): tablet c23f28e1595546559d274451d0394609 (table test-table [id=927b7170902f4e4e835a23b569cac402]) successfully deleted
I20260808 17:01:54.181764 3076 tablet_replica.cc:330] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963: stopping tablet replica
I20260808 17:01:54.181831 3076 raft_consensus.cc:2229] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:54.181861 3076 raft_consensus.cc:2258] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:54.182080 3076 ts_tablet_manager.cc:1905] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 17:01:54.183082 1925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:43947 (error 108)
I20260808 17:01:54.183364 3076 ts_tablet_manager.cc:1918] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 17:01:54.183427 3076 log.cc:1196] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/c23f28e1595546559d274451d0394609
I20260808 17:01:54.183457 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 2310
I20260808 17:01:54.183627 3076 ts_tablet_manager.cc:1939] T c23f28e1595546559d274451d0394609 P 754295157b374497891db284c0a93963: Deleting consensus metadata
I20260808 17:01:54.184145 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:54.183454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:43947 (local address 127.0.0.1:57080)
0808 17:01:54.184053 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:54.184104 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":58,"threads_started":1}
W20260808 17:01:54.184274 1925 catalog_manager.cc:4584] TS 351afd7919474272bc4431d70efb0987 (127.30.194.1:43947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c23f28e1595546559d274451d0394609: Network error: Client connection negotiation failed: client connection to 127.30.194.1:43947: connect: Connection refused (error 111)
W20260808 17:01:54.191875 3078 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:54.191737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:41279 (local address 127.0.0.1:57744)
0808 17:01:54.191790 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:54.191825 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.2:41279: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 17:01:54.192068 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 2535
I20260808 17:01:54.201269 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 1900
2026-08-08T17:01:54Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5949 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:54 dist-test-slave-2j7r krb5kdc[3087](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:54 dist-test-slave-2j7r krb5kdc[3087](info): set up 2 sockets
Aug 08 17:01:54 dist-test-slave-2j7r krb5kdc[3087](info): commencing operation
krb5kdc: starting...
W20260808 17:01:55.372123 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:01:55 dist-test-slave-2j7r krb5kdc[3087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:01:55.483032 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:39511
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:39573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:39511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:55.561925 3115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:55.562107 3115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:55.562129 3115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:55.563185 3115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:55.563218 3115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:55.563230 3115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:55.563241 3115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:55.563252 3115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:55.564618 3115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:39573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:39511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:39511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:55.564802 3115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:55.564968 3115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:55.569456 3132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:55.569521 3115 server_base.cc:1034] running on GCE node
W20260808 17:01:55.569481 3121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:55.569481 3122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:55.569875 3115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:55.570124 3115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:55.571266 3115 hybrid_clock.cc:648] HybridClock initialized: now 1786208515571247 us; error 33 us; skew 500 ppm
Aug 08 17:01:55 dist-test-slave-2j7r krb5kdc[3087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208515, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:55.573608 3115 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:01:55.574025 3115 webserver.cc:460] Webserver started at http://127.30.194.62:33343/ using document root <none> and password file <none>
I20260808 17:01:55.574235 3115 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:55.574287 3115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:55.574409 3115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:55.575347 3115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "877a0c238d624dd1b1939e57974bf076"
format_stamp: "Formatted at 2026-08-08 17:01:55 on dist-test-slave-2j7r"
server_key: "934bd304fdd0598f9d0a1530cc3058f3"
server_key_iv: "1dbecbb20425179ef395e14bb54a8881"
server_key_version: "encryptionkey@0"
I20260808 17:01:55.575651 3115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "877a0c238d624dd1b1939e57974bf076"
format_stamp: "Formatted at 2026-08-08 17:01:55 on dist-test-slave-2j7r"
server_key: "934bd304fdd0598f9d0a1530cc3058f3"
server_key_iv: "1dbecbb20425179ef395e14bb54a8881"
server_key_version: "encryptionkey@0"
I20260808 17:01:55.576830 3115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:01:55.577513 3151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:55.577658 3115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:55.577741 3115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "877a0c238d624dd1b1939e57974bf076"
format_stamp: "Formatted at 2026-08-08 17:01:55 on dist-test-slave-2j7r"
server_key: "934bd304fdd0598f9d0a1530cc3058f3"
server_key_iv: "1dbecbb20425179ef395e14bb54a8881"
server_key_version: "encryptionkey@0"
I20260808 17:01:55.577816 3115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:55.587560 3115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:55.589442 3115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:55.589566 3115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:55.593200 3115 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:39511
I20260808 17:01:55.593255 3235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:39511 every 8 connection(s)
I20260808 17:01:55.593538 3115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:55.594321 3236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:55.596500 3236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076: Bootstrap starting.
I20260808 17:01:55.597136 3236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:55.597188 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 3115
I20260808 17:01:55.597280 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:01:55.597432 31496 external_mini_cluster.cc:1428] Setting key b961f92ed7fa73a5b7203f1ae61a72d9
I20260808 17:01:55.597447 3236 log.cc:824] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:55.598285 3236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076: No bootstrap required, opened a new log
I20260808 17:01:55.600484 3236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877a0c238d624dd1b1939e57974bf076" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39511 } }
I20260808 17:01:55.600653 3236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:55.600713 3236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 877a0c238d624dd1b1939e57974bf076, State: Initialized, Role: FOLLOWER
I20260808 17:01:55.600843 3236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877a0c238d624dd1b1939e57974bf076" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39511 } }
I20260808 17:01:55.600927 3236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:55.600965 3236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:55.601012 3236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:55.601672 3236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877a0c238d624dd1b1939e57974bf076" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39511 } }
I20260808 17:01:55.601776 3236 leader_election.cc:302] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 877a0c238d624dd1b1939e57974bf076; no voters:
I20260808 17:01:55.601943 3236 leader_election.cc:288] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:55.601987 3253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:55.602159 3253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [term 1 LEADER]: Becoming Leader. State: Replica: 877a0c238d624dd1b1939e57974bf076, State: Running, Role: LEADER
I20260808 17:01:55.602205 3236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:55.602262 3253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877a0c238d624dd1b1939e57974bf076" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39511 } }
I20260808 17:01:55.602572 3253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 877a0c238d624dd1b1939e57974bf076. Latest consensus state: current_term: 1 leader_uuid: "877a0c238d624dd1b1939e57974bf076" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877a0c238d624dd1b1939e57974bf076" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39511 } } }
I20260808 17:01:55.602653 3253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:55.602818 3254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "877a0c238d624dd1b1939e57974bf076" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877a0c238d624dd1b1939e57974bf076" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39511 } } }
I20260808 17:01:55.602890 3254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:55.603019 3269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:55.603502 3269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:01:55 dist-test-slave-2j7r krb5kdc[3087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:55.605213 3269 catalog_manager.cc:1347] Generated new cluster ID: 5dfb61ee1916446b9d60b556733a18c1
I20260808 17:01:55.605268 3269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:55.608000 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:55.597853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.30.194.62:39511)
0808 17:01:55.598009 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:55.598016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:55.598041 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:01:55.598492 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:55.598499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:55.598545 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:55.598651 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:55.599469 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:55.600315 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:55.601374 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:55.601760 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:55.601868 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:55.604412 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:55.604416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:55.604416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:55.604428 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:55.606417 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:55.606805 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:55.606808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:55.606808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:55.606931 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:55.607114 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:55.607115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:55.607115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:55.607422 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:55.607585 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:55.607656 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 17:01:55.607790 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260808 17:01:55.615815 3269 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:55.616473 3269 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:55.621351 3269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 877a0c238d624dd1b1939e57974bf076: Generated new TSK 0
I20260808 17:01:55.621515 3269 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:01:55.647287 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:39511
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:39573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:55.726996 3289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:55.727173 3289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:55.727193 3289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:55.728303 3289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:55.728340 3289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:55.728367 3289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:01:55.729789 3289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:39573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:55.730003 3289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:55.730197 3289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:55.734867 3312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:55.734819 3304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:55.734833 3302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:55.735069 3289 server_base.cc:1034] running on GCE node
I20260808 17:01:55.735252 3289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:55.735452 3289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:55.736586 3289 hybrid_clock.cc:648] HybridClock initialized: now 1786208515736577 us; error 27 us; skew 500 ppm
Aug 08 17:01:55 dist-test-slave-2j7r krb5kdc[3087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208515, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:55.738880 3289 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:01:55.739249 3289 webserver.cc:460] Webserver started at http://127.30.194.1:41309/ using document root <none> and password file <none>
I20260808 17:01:55.739486 3289 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:55.739533 3289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:55.739656 3289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:55.740597 3289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "e97fedabc63e4ffb97dc94e8445a5572"
format_stamp: "Formatted at 2026-08-08 17:01:55 on dist-test-slave-2j7r"
server_key: "e81acebcbd6637306f10276d4eb6f9ba"
server_key_iv: "93fc7dae8a4fc7f4b78441316f501c2b"
server_key_version: "encryptionkey@0"
I20260808 17:01:55.740899 3289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "e97fedabc63e4ffb97dc94e8445a5572"
format_stamp: "Formatted at 2026-08-08 17:01:55 on dist-test-slave-2j7r"
server_key: "e81acebcbd6637306f10276d4eb6f9ba"
server_key_iv: "93fc7dae8a4fc7f4b78441316f501c2b"
server_key_version: "encryptionkey@0"
I20260808 17:01:55.742089 3289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 17:01:55.742755 3321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:55.742897 3289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:55.742975 3289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "e97fedabc63e4ffb97dc94e8445a5572"
format_stamp: "Formatted at 2026-08-08 17:01:55 on dist-test-slave-2j7r"
server_key: "e81acebcbd6637306f10276d4eb6f9ba"
server_key_iv: "93fc7dae8a4fc7f4b78441316f501c2b"
server_key_version: "encryptionkey@0"
I20260808 17:01:55.743038 3289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:55.756631 3289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:55.758319 3289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:55.758441 3289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:55.758620 3289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:55.758885 3289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:55.758919 3289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:55.758944 3289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:55.758960 3289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:55.764209 3289 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:35863
I20260808 17:01:55.764289 3522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:35863 every 8 connection(s)
I20260808 17:01:55.764530 3289 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:01:55 dist-test-slave-2j7r krb5kdc[3087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208515, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:55.771638 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 3289
I20260808 17:01:55.771734 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:01:55.771926 31496 external_mini_cluster.cc:1428] Setting key c230e496974c1d1a453a0d47649cd390
I20260808 17:01:55.773114 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:55.765185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:54457 (local address 127.30.194.62:39511)
0808 17:01:55.765224 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:55.765226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:55.765488 (+ 262us) server_negotiation.cc:407] Connection header received
0808 17:01:55.765946 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:55.765948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:55.765961 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:55.766008 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:55.766908 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:55.767415 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:55.768345 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:55.768557 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:55.768612 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:55.770563 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:55.770565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:55.770565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:55.770573 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:55.771663 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:55.772148 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:55.772149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:55.772149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:55.772212 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:55.772457 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:55.772458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:55.772458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:55.772587 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:55.772655 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:55.772962 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 17:01:55.773030 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:01:55.773386 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:55.765228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:39511 (local address 127.30.194.1:54457)
0808 17:01:55.765377 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:55.765410 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:55.765845 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:55.766084 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:55.766088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:55.766634 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:55.766817 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:55.766822 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:55.767494 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:55.767495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:55.768262 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:55.768264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:55.768624 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:55.768625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:55.768741 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:55.769204 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:55.769212 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:55.770481 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:55.771753 (+ 1272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:55.771761 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:55.771763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:55.772059 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:55.772281 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:55.772282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:55.772282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:55.772395 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:55.772660 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:55.772667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:55.772824 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 17:01:55.772951 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 17:01:55.773126 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260808 17:01:55.773880 3523 heartbeater.cc:347] Connected to a master server at 127.30.194.62:39511
I20260808 17:01:55.773964 3523 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:55.774118 3523 heartbeater.cc:511] Master 127.30.194.62:39511 requested a full tablet report, sending...
I20260808 17:01:55.774597 3180 ts_manager.cc:194] Registered new tserver with Master: e97fedabc63e4ffb97dc94e8445a5572 (127.30.194.1:35863)
I20260808 17:01:55.775161 3180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:54457
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:01:55.795706 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:39511
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:39573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:55.876060 3528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:55.876242 3528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:55.876264 3528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:55.877394 3528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:55.877434 3528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:55.877458 3528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:01:55.878813 3528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:39573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:55.879014 3528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:55.879197 3528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:55.883699 3539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:55.883718 3536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:55.883761 3541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:55.884305 3528 server_base.cc:1034] running on GCE node
I20260808 17:01:55.884493 3528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:55.884711 3528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:55.885854 3528 hybrid_clock.cc:648] HybridClock initialized: now 1786208515885837 us; error 31 us; skew 500 ppm
Aug 08 17:01:55 dist-test-slave-2j7r krb5kdc[3087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208515, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:55.888065 3528 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:01:55.888487 3528 webserver.cc:460] Webserver started at http://127.30.194.2:35197/ using document root <none> and password file <none>
I20260808 17:01:55.888679 3528 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:55.888722 3528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:55.888835 3528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:55.889814 3528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "c6a7c5ba917a4a22ac6fed46fb748e37"
format_stamp: "Formatted at 2026-08-08 17:01:55 on dist-test-slave-2j7r"
server_key: "517bc8fa042db609ea838a1e54b48066"
server_key_iv: "30ae68d85a4cd1dfcceef21873bc182c"
server_key_version: "encryptionkey@0"
I20260808 17:01:55.890120 3528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "c6a7c5ba917a4a22ac6fed46fb748e37"
format_stamp: "Formatted at 2026-08-08 17:01:55 on dist-test-slave-2j7r"
server_key: "517bc8fa042db609ea838a1e54b48066"
server_key_iv: "30ae68d85a4cd1dfcceef21873bc182c"
server_key_version: "encryptionkey@0"
I20260808 17:01:55.891317 3528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:55.891933 3550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:55.892087 3528 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:55.892176 3528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "c6a7c5ba917a4a22ac6fed46fb748e37"
format_stamp: "Formatted at 2026-08-08 17:01:55 on dist-test-slave-2j7r"
server_key: "517bc8fa042db609ea838a1e54b48066"
server_key_iv: "30ae68d85a4cd1dfcceef21873bc182c"
server_key_version: "encryptionkey@0"
I20260808 17:01:55.892261 3528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:55.912808 3528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:55.914366 3528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:55.914487 3528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:55.914665 3528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:55.914927 3528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:55.914959 3528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:55.914984 3528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:55.915001 3528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:55.920363 3528 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:38725
I20260808 17:01:55.920418 3770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:38725 every 8 connection(s)
I20260808 17:01:55.920673 3528 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:01:55 dist-test-slave-2j7r krb5kdc[3087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208515, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:55.928560 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:55.921309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:38599 (local address 127.30.194.62:39511)
0808 17:01:55.921366 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:55.921369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:55.921546 (+ 177us) server_negotiation.cc:407] Connection header received
0808 17:01:55.922003 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:55.922005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:55.922018 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:55.922068 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:55.922920 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:55.923419 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:55.924369 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:55.924565 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:55.924621 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:55.926504 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:55.926505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:55.926506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:55.926512 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:55.927540 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:55.927886 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:55.927887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:55.927887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:55.927930 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:55.928099 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:55.928099 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:55.928100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:55.928188 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:55.928255 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:55.928442 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 17:01:55.928498 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:01:55.928692 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:55.921262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:39511 (local address 127.30.194.2:38599)
0808 17:01:55.921428 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:55.921467 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:55.921912 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:55.922132 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:55.922135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:55.922652 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:55.922830 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:55.922834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:55.923496 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:55.923497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:55.924281 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:55.924283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:55.924653 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:55.924654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:55.924766 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:55.925175 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:55.925182 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:55.926423 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:55.927609 (+ 1186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:55.927615 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:55.927616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:55.927830 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:55.927973 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:55.927973 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:55.927973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:55.928059 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:55.928248 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:55.928250 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:55.928350 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 17:01:55.928421 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:01:55.928527 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260808 17:01:55.929144 3771 heartbeater.cc:347] Connected to a master server at 127.30.194.62:39511
I20260808 17:01:55.929247 3771 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:55.929379 3771 heartbeater.cc:511] Master 127.30.194.62:39511 requested a full tablet report, sending...
I20260808 17:01:55.929682 3180 ts_manager.cc:194] Registered new tserver with Master: c6a7c5ba917a4a22ac6fed46fb748e37 (127.30.194.2:38725)
I20260808 17:01:55.930022 3180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:38599
I20260808 17:01:55.930356 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 3528
I20260808 17:01:55.930457 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:01:55.930653 31496 external_mini_cluster.cc:1428] Setting key 7b51e2d02e079c23c0a9a0347e9eaa4c
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:01:55.953819 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:39511
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:39573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:56.031699 3780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:56.031890 3780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:56.031912 3780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:56.032975 3780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:56.033005 3780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:56.033030 3780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:01:56.034375 3780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:39573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:39511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:56.034547 3780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:56.034711 3780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:56.039242 3797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:56.039274 3789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:56.039443 3780 server_base.cc:1034] running on GCE node
W20260808 17:01:56.039340 3790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:56.039664 3780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:56.039868 3780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:56.041002 3780 hybrid_clock.cc:648] HybridClock initialized: now 1786208516040992 us; error 26 us; skew 500 ppm
Aug 08 17:01:56 dist-test-slave-2j7r krb5kdc[3087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:56.043349 3780 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:01:56.043733 3780 webserver.cc:460] Webserver started at http://127.30.194.3:46289/ using document root <none> and password file <none>
I20260808 17:01:56.043949 3780 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:56.043991 3780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:56.044108 3780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:56.045054 3780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "8296e839c2b349bf849260a6c9904ea3"
format_stamp: "Formatted at 2026-08-08 17:01:56 on dist-test-slave-2j7r"
server_key: "9254030aca31a99b95317113aeb46f2a"
server_key_iv: "455d9648cd7e4b7475451f09d3de3f19"
server_key_version: "encryptionkey@0"
I20260808 17:01:56.045410 3780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "8296e839c2b349bf849260a6c9904ea3"
format_stamp: "Formatted at 2026-08-08 17:01:56 on dist-test-slave-2j7r"
server_key: "9254030aca31a99b95317113aeb46f2a"
server_key_iv: "455d9648cd7e4b7475451f09d3de3f19"
server_key_version: "encryptionkey@0"
I20260808 17:01:56.046653 3780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:56.047345 3808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:56.047480 3780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:56.047564 3780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "8296e839c2b349bf849260a6c9904ea3"
format_stamp: "Formatted at 2026-08-08 17:01:56 on dist-test-slave-2j7r"
server_key: "9254030aca31a99b95317113aeb46f2a"
server_key_iv: "455d9648cd7e4b7475451f09d3de3f19"
server_key_version: "encryptionkey@0"
I20260808 17:01:56.047629 3780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:56.065882 3780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:56.067646 3780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:56.067782 3780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:56.067974 3780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:56.068251 3780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:56.068284 3780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:56.068311 3780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:56.068328 3780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:56.073769 3780 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:40089
I20260808 17:01:56.073836 4025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:40089 every 8 connection(s)
I20260808 17:01:56.074092 3780 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:01:56.077827 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 3780
I20260808 17:01:56.077930 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:01:56.078102 31496 external_mini_cluster.cc:1428] Setting key b87e2920e01b83b1bf1b5b39849e4500
Aug 08 17:01:56 dist-test-slave-2j7r krb5kdc[3087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:56.082855 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:56.074702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:58149 (local address 127.30.194.62:39511)
0808 17:01:56.074765 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:56.074768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:56.074957 (+ 189us) server_negotiation.cc:407] Connection header received
0808 17:01:56.075427 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:56.075429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:56.075442 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:56.075492 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:56.076370 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:56.076871 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:56.077932 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:56.078194 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:56.078261 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:56.080474 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:56.080476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:56.080477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:56.080485 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:56.081731 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:56.082117 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:56.082118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:56.082118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:56.082162 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:56.082378 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:56.082379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:56.082379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:56.082469 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:56.082511 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:56.082728 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 17:01:56.082792 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:01:56.082986 4030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:56.074654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:39511 (local address 127.30.194.3:58149)
0808 17:01:56.074845 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:56.074880 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:56.075332 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:56.075558 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:56.075562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:56.076082 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:56.076278 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:56.076283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:56.076954 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:56.076956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:56.077851 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:56.077854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:56.078278 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:56.078280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:56.078429 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:56.078922 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:56.078932 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:56.080364 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:56.081816 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:56.081822 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:56.081824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:56.082051 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:56.082231 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:56.082231 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:56.082231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:56.082329 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:56.082513 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:56.082516 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:56.082627 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 17:01:56.082708 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:01:56.082825 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
I20260808 17:01:56.083457 4027 heartbeater.cc:347] Connected to a master server at 127.30.194.62:39511
I20260808 17:01:56.083541 4027 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:56.083691 4027 heartbeater.cc:511] Master 127.30.194.62:39511 requested a full tablet report, sending...
I20260808 17:01:56.084018 3180 ts_manager.cc:194] Registered new tserver with Master: 8296e839c2b349bf849260a6c9904ea3 (127.30.194.3:40089)
I20260808 17:01:56.084374 3180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:58149
I20260808 17:01:56.089649 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 17:01:56.101228 3251 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:56.100403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43258 (local address 127.30.194.62:39511)
0808 17:01:56.100453 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:56.100455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:56.100555 (+ 100us) server_negotiation.cc:407] Connection header received
0808 17:01:56.100595 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:56.100596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:56.100604 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:56.100647 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:56.101167 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43258: BlockingRecv error: recv got EOF from 127.0.0.1:43258 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:01:56.102613 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 3289
I20260808 17:01:56.128489 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 3528
I20260808 17:01:56.134541 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 3780
I20260808 17:01:56.143935 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 3115
2026-08-08T17:01:56Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1939 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:56 dist-test-slave-2j7r krb5kdc[4048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:56 dist-test-slave-2j7r krb5kdc[4048](info): set up 2 sockets
Aug 08 17:01:56 dist-test-slave-2j7r krb5kdc[4048](info): commencing operation
krb5kdc: starting...
W20260808 17:01:57.309621 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:01:57 dist-test-slave-2j7r krb5kdc[4048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:01:57.419481 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40463
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:40463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:57.499364 4069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:57.499557 4069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:57.499581 4069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:57.500662 4069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:57.500698 4069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:57.500713 4069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:57.500726 4069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:57.500739 4069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:57.502193 4069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:40463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:57.502382 4069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:57.502561 4069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:57.506946 4082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:57.506927 4079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:57.507054 4077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:57.507201 4069 server_base.cc:1034] running on GCE node
I20260808 17:01:57.507428 4069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:57.507685 4069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:57.508828 4069 hybrid_clock.cc:648] HybridClock initialized: now 1786208517508810 us; error 33 us; skew 500 ppm
Aug 08 17:01:57 dist-test-slave-2j7r krb5kdc[4048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208517, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:57.511155 4069 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:01:57.511510 4069 webserver.cc:460] Webserver started at http://127.30.194.62:45519/ using document root <none> and password file <none>
I20260808 17:01:57.511690 4069 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:57.511730 4069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:57.511816 4069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:57.512701 4069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc"
format_stamp: "Formatted at 2026-08-08 17:01:57 on dist-test-slave-2j7r"
server_key: "c38de04ef1c44d549955c6fcc2bbeb10"
server_key_iv: "2fdd7c8c5125c6a68b1599cba7cab5e5"
server_key_version: "encryptionkey@0"
I20260808 17:01:57.512965 4069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc"
format_stamp: "Formatted at 2026-08-08 17:01:57 on dist-test-slave-2j7r"
server_key: "c38de04ef1c44d549955c6fcc2bbeb10"
server_key_iv: "2fdd7c8c5125c6a68b1599cba7cab5e5"
server_key_version: "encryptionkey@0"
I20260808 17:01:57.514194 4069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:57.514858 4096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:57.515029 4069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:57.515113 4069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc"
format_stamp: "Formatted at 2026-08-08 17:01:57 on dist-test-slave-2j7r"
server_key: "c38de04ef1c44d549955c6fcc2bbeb10"
server_key_iv: "2fdd7c8c5125c6a68b1599cba7cab5e5"
server_key_version: "encryptionkey@0"
I20260808 17:01:57.515167 4069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:57.532923 4069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:57.534869 4069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:57.534973 4069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:57.538630 4069 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:40463
I20260808 17:01:57.538671 4198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:40463 every 8 connection(s)
I20260808 17:01:57.539172 4069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:57.539812 4200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:57.541997 4200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc: Bootstrap starting.
I20260808 17:01:57.542538 4200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:57.542791 4200 log.cc:824] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:57.543440 4200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc: No bootstrap required, opened a new log
I20260808 17:01:57.543613 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 4069
I20260808 17:01:57.543705 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:01:57.543869 31496 external_mini_cluster.cc:1428] Setting key e9a7ca64dbee677eb37fecd6e891c13a
I20260808 17:01:57.545780 4200 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40463 } }
I20260808 17:01:57.545912 4200 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:57.545997 4200 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb9b2d2f9ff646c9a512a0c79fb413dc, State: Initialized, Role: FOLLOWER
I20260808 17:01:57.546141 4200 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40463 } }
I20260808 17:01:57.546231 4200 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:57.546270 4200 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:57.546317 4200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:57.546895 4200 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40463 } }
I20260808 17:01:57.546984 4200 leader_election.cc:302] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb9b2d2f9ff646c9a512a0c79fb413dc; no voters:
I20260808 17:01:57.547124 4200 leader_election.cc:288] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:57.547168 4208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:57.547312 4208 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [term 1 LEADER]: Becoming Leader. State: Replica: cb9b2d2f9ff646c9a512a0c79fb413dc, State: Running, Role: LEADER
I20260808 17:01:57.547394 4200 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:57.547449 4208 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40463 } }
I20260808 17:01:57.547787 4208 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb9b2d2f9ff646c9a512a0c79fb413dc. Latest consensus state: current_term: 1 leader_uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40463 } } }
I20260808 17:01:57.547864 4208 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:57.548005 4211 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb9b2d2f9ff646c9a512a0c79fb413dc" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40463 } } }
I20260808 17:01:57.548064 4211 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:57.548214 4218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:57.549144 4218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:57.551123 4218 catalog_manager.cc:1347] Generated new cluster ID: 27bd773bef7547d3b61503a73790e668
I20260808 17:01:57.551220 4218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:01:57 dist-test-slave-2j7r krb5kdc[4048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:57.554706 4207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:57.544311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50778 (local address 127.30.194.62:40463)
0808 17:01:57.544478 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:57.544487 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:57.544510 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:01:57.545762 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:57.545770 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:57.545821 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:57.545949 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:57.546678 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:57.547883 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:57.548830 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:57.549541 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:57.549656 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:57.551930 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:57.551934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:57.551935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:57.551946 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:57.553393 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:57.553698 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:57.553701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:57.553701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:57.553806 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:57.553981 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:57.553981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:57.553982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:57.554199 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:57.554341 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:57.554444 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 17:01:57.554569 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260808 17:01:57.555496 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:57.565964 4218 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:57.566627 4218 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:57.573128 4218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb9b2d2f9ff646c9a512a0c79fb413dc: Generated new TSK 0
I20260808 17:01:57.573275 4218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:57.577898 4207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:57.574314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50790 (local address 127.30.194.62:40463)
0808 17:01:57.574355 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:57.574357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:57.574500 (+ 143us) server_negotiation.cc:407] Connection header received
0808 17:01:57.574560 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:57.574561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:57.574568 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:57.574616 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:57.574988 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:57.575452 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:57.576504 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:57.576739 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:57.576805 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:57.577501 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:57.577502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:57.577503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:57.577664 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:57.577679 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:57.577697 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:57.577745 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:57.577815 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 17:01:57.577838 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:01:57.580271 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 4069
2026-08-08T17:01:57Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1445 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:57 dist-test-slave-2j7r krb5kdc[4253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:57 dist-test-slave-2j7r krb5kdc[4253](info): set up 2 sockets
Aug 08 17:01:57 dist-test-slave-2j7r krb5kdc[4253](info): commencing operation
krb5kdc: starting...
W20260808 17:01:58.729615 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:01:58 dist-test-slave-2j7r krb5kdc[4253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:01:58.838883 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:34597
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:37187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:34597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:58.920878 4280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:58.921059 4280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:58.921104 4280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:58.922195 4280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:58.922235 4280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:58.922253 4280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:58.922268 4280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:58.922283 4280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:58.923697 4280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:37187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:34597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:34597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:58.923925 4280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:58.924124 4280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:58.928609 4286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:58.928637 4287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:58.928642 4293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:58.928658 4280 server_base.cc:1034] running on GCE node
I20260808 17:01:58.928961 4280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:58.929248 4280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:58.930392 4280 hybrid_clock.cc:648] HybridClock initialized: now 1786208518930375 us; error 30 us; skew 500 ppm
Aug 08 17:01:58 dist-test-slave-2j7r krb5kdc[4253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208518, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:58.932914 4280 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:01:58.933326 4280 webserver.cc:460] Webserver started at http://127.30.194.62:46067/ using document root <none> and password file <none>
I20260808 17:01:58.933534 4280 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:58.933581 4280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:58.933701 4280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:58.934652 4280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "8c9a3c01a80a40fbb067d34616c0c430"
format_stamp: "Formatted at 2026-08-08 17:01:58 on dist-test-slave-2j7r"
server_key: "a1600b82690cbb9b770ef6e54a2e9170"
server_key_iv: "afe7f510f7144fe5cd307f192f31914f"
server_key_version: "encryptionkey@0"
I20260808 17:01:58.934958 4280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "8c9a3c01a80a40fbb067d34616c0c430"
format_stamp: "Formatted at 2026-08-08 17:01:58 on dist-test-slave-2j7r"
server_key: "a1600b82690cbb9b770ef6e54a2e9170"
server_key_iv: "afe7f510f7144fe5cd307f192f31914f"
server_key_version: "encryptionkey@0"
I20260808 17:01:58.936084 4280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:58.936669 4303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:58.936813 4280 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:58.936894 4280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "8c9a3c01a80a40fbb067d34616c0c430"
format_stamp: "Formatted at 2026-08-08 17:01:58 on dist-test-slave-2j7r"
server_key: "a1600b82690cbb9b770ef6e54a2e9170"
server_key_iv: "afe7f510f7144fe5cd307f192f31914f"
server_key_version: "encryptionkey@0"
I20260808 17:01:58.936954 4280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:58.951285 4280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:58.952756 4280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:58.952872 4280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:58.956470 4280 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:34597
I20260808 17:01:58.956509 4388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:34597 every 8 connection(s)
I20260808 17:01:58.956786 4280 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:58.957768 4390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:58.959787 4390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430: Bootstrap starting.
I20260808 17:01:58.960299 4390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:58.960544 4390 log.cc:824] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:58.961179 4390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430: No bootstrap required, opened a new log
I20260808 17:01:58.962723 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 4280
I20260808 17:01:58.962677 4390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9a3c01a80a40fbb067d34616c0c430" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 34597 } }
I20260808 17:01:58.962800 4390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:58.962819 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:01:58.962842 4390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9a3c01a80a40fbb067d34616c0c430, State: Initialized, Role: FOLLOWER
I20260808 17:01:58.962976 31496 external_mini_cluster.cc:1428] Setting key 8b4a21a8432691b15d24dccf6004bb5a
I20260808 17:01:58.962973 4390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9a3c01a80a40fbb067d34616c0c430" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 34597 } }
I20260808 17:01:58.963055 4390 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:58.963094 4390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:58.963140 4390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:58.964093 4390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9a3c01a80a40fbb067d34616c0c430" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 34597 } }
I20260808 17:01:58.964182 4390 leader_election.cc:302] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c9a3c01a80a40fbb067d34616c0c430; no voters:
I20260808 17:01:58.964329 4390 leader_election.cc:288] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:58.964372 4397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:58.964524 4397 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [term 1 LEADER]: Becoming Leader. State: Replica: 8c9a3c01a80a40fbb067d34616c0c430, State: Running, Role: LEADER
I20260808 17:01:58.964566 4390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:58.964615 4397 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9a3c01a80a40fbb067d34616c0c430" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 34597 } }
I20260808 17:01:58.964948 4397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c9a3c01a80a40fbb067d34616c0c430. Latest consensus state: current_term: 1 leader_uuid: "8c9a3c01a80a40fbb067d34616c0c430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9a3c01a80a40fbb067d34616c0c430" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 34597 } } }
I20260808 17:01:58.964958 4399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c9a3c01a80a40fbb067d34616c0c430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9a3c01a80a40fbb067d34616c0c430" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 34597 } } }
I20260808 17:01:58.965162 4399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:58.965344 4397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:58.965428 4410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:58.965925 4410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:58.967607 4410 catalog_manager.cc:1347] Generated new cluster ID: dd375bf8b57f490c8afc6e0f5a373454
I20260808 17:01:58.967654 4410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:01:58 dist-test-slave-2j7r krb5kdc[4253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:58.972833 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:58.963399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.30.194.62:34597)
0808 17:01:58.963596 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:58.963606 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:58.963642 (+ 36us) server_negotiation.cc:407] Connection header received
0808 17:01:58.963769 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:58.963776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:58.963832 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:58.963970 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:58.964612 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:58.965954 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:58.967009 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:58.967519 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:58.967631 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:58.969764 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:58.969768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:58.969769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:58.969788 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:58.971238 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:58.971548 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:58.971551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:58.971551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:58.971638 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:58.971809 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:58.971810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:58.971810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:58.972285 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:58.972423 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:58.972531 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 17:01:58.972656 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260808 17:01:58.980700 4410 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:58.981393 4410 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:58.988603 4410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c9a3c01a80a40fbb067d34616c0c430: Generated new TSK 0
I20260808 17:01:58.988762 4410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:01:59.017488 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:34597
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:37187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:59.096381 4428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:59.096562 4428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:59.096583 4428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:59.097648 4428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:59.097680 4428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:59.097704 4428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:01:59.099064 4428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:37187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:34597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:59.099246 4428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:59.099409 4428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:59.103960 4436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:59.104019 4428 server_base.cc:1034] running on GCE node
W20260808 17:01:59.103955 4442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:59.103956 4437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:59.104337 4428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:59.104566 4428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:59.105698 4428 hybrid_clock.cc:648] HybridClock initialized: now 1786208519105688 us; error 31 us; skew 500 ppm
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[4253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:59.107946 4428 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:01:59.108319 4428 webserver.cc:460] Webserver started at http://127.30.194.1:42223/ using document root <none> and password file <none>
I20260808 17:01:59.108517 4428 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:59.108564 4428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:59.108685 4428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:59.109700 4428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "3abdd25556c84457a085ba9bf4d5d716"
format_stamp: "Formatted at 2026-08-08 17:01:59 on dist-test-slave-2j7r"
server_key: "3143934b164fcc6417535e9e4c9050b8"
server_key_iv: "1903b37be4455937141c7f6092acd87b"
server_key_version: "encryptionkey@0"
I20260808 17:01:59.110045 4428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "3abdd25556c84457a085ba9bf4d5d716"
format_stamp: "Formatted at 2026-08-08 17:01:59 on dist-test-slave-2j7r"
server_key: "3143934b164fcc6417535e9e4c9050b8"
server_key_iv: "1903b37be4455937141c7f6092acd87b"
server_key_version: "encryptionkey@0"
I20260808 17:01:59.111222 4428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:59.111845 4453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:59.111985 4428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:59.112066 4428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "3abdd25556c84457a085ba9bf4d5d716"
format_stamp: "Formatted at 2026-08-08 17:01:59 on dist-test-slave-2j7r"
server_key: "3143934b164fcc6417535e9e4c9050b8"
server_key_iv: "1903b37be4455937141c7f6092acd87b"
server_key_version: "encryptionkey@0"
I20260808 17:01:59.112128 4428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:59.123497 4428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:59.124977 4428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:59.125121 4428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:59.125324 4428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:59.125591 4428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:59.125623 4428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:59.125648 4428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:59.125672 4428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:59.130980 4428 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:45025
I20260808 17:01:59.131040 4662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:45025 every 8 connection(s)
I20260808 17:01:59.131279 4428 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:01:59.131407 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 4428
I20260808 17:01:59.131513 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:01:59.131703 31496 external_mini_cluster.cc:1428] Setting key 1b69b9613c65e64e3d7974b466ba7a92
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[4253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
I20260808 17:01:59.140398 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:59.131951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:33311 (local address 127.30.194.62:34597)
0808 17:01:59.132000 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:59.132002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:59.132207 (+ 205us) server_negotiation.cc:407] Connection header received
0808 17:01:59.132732 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:59.132732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:59.132743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:59.132780 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:59.133872 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.134586 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.135524 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.135728 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.135783 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:59.137795 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:59.137797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:59.137797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:59.137804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:59.139044 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:59.139454 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:59.139456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:59.139456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:59.139515 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:59.139724 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:59.139725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:59.139726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:59.139837 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:59.139902 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:59.140182 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 17:01:59.140279 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:01:59.140497 4666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:59.131958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:34597 (local address 127.30.194.1:33311)
0808 17:01:59.132107 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:59.132144 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:59.132641 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:59.132842 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:59.132846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:59.133552 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:59.133791 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:59.133797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.134676 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.134677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:59.135441 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:59.135443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.135797 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.135798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:59.135920 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:59.136360 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:59.136367 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:59.137710 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:59.139108 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:59.139115 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:59.139118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:59.139391 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:59.139566 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:59.139567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:59.139567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:59.139670 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:59.139891 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:59.139898 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:59.140030 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 17:01:59.140154 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 17:01:59.140290 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260808 17:01:59.140957 4664 heartbeater.cc:347] Connected to a master server at 127.30.194.62:34597
I20260808 17:01:59.141043 4664 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:59.141227 4664 heartbeater.cc:511] Master 127.30.194.62:34597 requested a full tablet report, sending...
I20260808 17:01:59.141691 4332 ts_manager.cc:194] Registered new tserver with Master: 3abdd25556c84457a085ba9bf4d5d716 (127.30.194.1:45025)
I20260808 17:01:59.142323 4332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:33311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:01:59.155366 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:34597
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:37187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:59.233706 4669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:59.233899 4669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:59.233929 4669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:59.235020 4669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:59.235060 4669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:59.235092 4669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:01:59.236496 4669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:37187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:34597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:59.236729 4669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:59.236930 4669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:59.241640 4681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:59.241648 4679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:59.241653 4677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:59.241848 4669 server_base.cc:1034] running on GCE node
I20260808 17:01:59.242098 4669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:59.242327 4669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:59.243481 4669 hybrid_clock.cc:648] HybridClock initialized: now 1786208519243471 us; error 29 us; skew 500 ppm
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[4253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:59.245844 4669 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:01:59.246258 4669 webserver.cc:460] Webserver started at http://127.30.194.2:44181/ using document root <none> and password file <none>
I20260808 17:01:59.246465 4669 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:59.246510 4669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:59.246630 4669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:59.247581 4669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "02e4075b959342e8a565e4d1fefa50cb"
format_stamp: "Formatted at 2026-08-08 17:01:59 on dist-test-slave-2j7r"
server_key: "b7419ddd9de20fc9b11c6898b5e6e855"
server_key_iv: "25a9c70bee60e80330b205a312843463"
server_key_version: "encryptionkey@0"
I20260808 17:01:59.247916 4669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "02e4075b959342e8a565e4d1fefa50cb"
format_stamp: "Formatted at 2026-08-08 17:01:59 on dist-test-slave-2j7r"
server_key: "b7419ddd9de20fc9b11c6898b5e6e855"
server_key_iv: "25a9c70bee60e80330b205a312843463"
server_key_version: "encryptionkey@0"
I20260808 17:01:59.249171 4669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:59.249960 4694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:59.250135 4669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:59.250206 4669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "02e4075b959342e8a565e4d1fefa50cb"
format_stamp: "Formatted at 2026-08-08 17:01:59 on dist-test-slave-2j7r"
server_key: "b7419ddd9de20fc9b11c6898b5e6e855"
server_key_iv: "25a9c70bee60e80330b205a312843463"
server_key_version: "encryptionkey@0"
I20260808 17:01:59.250265 4669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:59.269430 4669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:59.271282 4669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:59.271397 4669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:59.271580 4669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:59.271837 4669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:59.271867 4669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:59.271893 4669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:59.271909 4669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:59.277364 4669 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:39409
I20260808 17:01:59.277434 4867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:39409 every 8 connection(s)
I20260808 17:01:59.277715 4669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:01:59.279542 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 4669
I20260808 17:01:59.279667 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:01:59.279851 31496 external_mini_cluster.cc:1428] Setting key 9d6bb7f7b7c825e39b3642b29fccc27f
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[4253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:59.287757 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:59.278441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:60197 (local address 127.30.194.62:34597)
0808 17:01:59.278518 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:59.278519 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:59.278704 (+ 185us) server_negotiation.cc:407] Connection header received
0808 17:01:59.279150 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:59.279151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:59.279161 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:59.279213 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:59.280317 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.280898 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.282286 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.282571 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.282634 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:59.284856 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:59.284858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:59.284858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:59.284866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:59.286389 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:59.286774 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:59.286775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:59.286776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:59.286838 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:59.287093 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:59.287094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:59.287095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:59.287238 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:59.287309 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:59.287574 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 17:01:59.287663 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 17:01:59.287848 4871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:59.278402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:34597 (local address 127.30.194.2:60197)
0808 17:01:59.278561 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:59.278603 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:59.279069 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:59.279278 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:59.279281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:59.280017 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:59.280237 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:59.280242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.280995 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.280997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:59.282165 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:59.282168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.282652 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.282652 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:59.282757 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:59.283258 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:59.283266 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:59.284759 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:59.286474 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:59.286481 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:59.286482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:59.286694 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:59.286917 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:59.286918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:59.286918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:59.287025 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:59.287323 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:59.287330 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:59.287448 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 17:01:59.287540 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 17:01:59.287660 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260808 17:01:59.288364 4868 heartbeater.cc:347] Connected to a master server at 127.30.194.62:34597
I20260808 17:01:59.288463 4868 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:59.288637 4868 heartbeater.cc:511] Master 127.30.194.62:34597 requested a full tablet report, sending...
I20260808 17:01:59.289026 4332 ts_manager.cc:194] Registered new tserver with Master: 02e4075b959342e8a565e4d1fefa50cb (127.30.194.2:39409)
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
I20260808 17:01:59.289549 4332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:60197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:01:59.305593 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:34597
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:37187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:59.385735 4880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:59.385911 4880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:59.385947 4880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:59.386997 4880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:59.387028 4880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:59.387053 4880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:01:59.388394 4880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:37187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:34597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:59.388567 4880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:59.388736 4880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:59.393343 4888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:59.393348 4889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:59.393347 4891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:59.393540 4880 server_base.cc:1034] running on GCE node
I20260808 17:01:59.393765 4880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:59.393991 4880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:59.395123 4880 hybrid_clock.cc:648] HybridClock initialized: now 1786208519395106 us; error 30 us; skew 500 ppm
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[4253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:59.397432 4880 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:01:59.397807 4880 webserver.cc:460] Webserver started at http://127.30.194.3:36013/ using document root <none> and password file <none>
I20260808 17:01:59.398001 4880 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:59.398047 4880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:59.398172 4880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:59.399085 4880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "d9e7ef3347ac4804b450213b8d5032a2"
format_stamp: "Formatted at 2026-08-08 17:01:59 on dist-test-slave-2j7r"
server_key: "3de071608ba56dc0f1b801876bf2540b"
server_key_iv: "513b5525e951d309f8e1430e82fb38c4"
server_key_version: "encryptionkey@0"
I20260808 17:01:59.399403 4880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "d9e7ef3347ac4804b450213b8d5032a2"
format_stamp: "Formatted at 2026-08-08 17:01:59 on dist-test-slave-2j7r"
server_key: "3de071608ba56dc0f1b801876bf2540b"
server_key_iv: "513b5525e951d309f8e1430e82fb38c4"
server_key_version: "encryptionkey@0"
I20260808 17:01:59.400552 4880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:01:59.401176 4905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:59.401314 4880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:59.401396 4880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "d9e7ef3347ac4804b450213b8d5032a2"
format_stamp: "Formatted at 2026-08-08 17:01:59 on dist-test-slave-2j7r"
server_key: "3de071608ba56dc0f1b801876bf2540b"
server_key_iv: "513b5525e951d309f8e1430e82fb38c4"
server_key_version: "encryptionkey@0"
I20260808 17:01:59.401458 4880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:59.423007 4880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:59.424906 4880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:59.425030 4880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:59.425241 4880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:59.425532 4880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:59.425565 4880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:59.425592 4880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:59.425616 4880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:59.430967 4880 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:43209
I20260808 17:01:59.431032 5123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:43209 every 8 connection(s)
I20260808 17:01:59.431289 4880 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[4253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:59.439435 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:59.431929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:37689 (local address 127.30.194.62:34597)
0808 17:01:59.431991 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:59.431993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:59.432147 (+ 154us) server_negotiation.cc:407] Connection header received
0808 17:01:59.432640 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:59.432642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:59.432653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:59.432728 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:59.433560 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.434047 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.434979 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.435169 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.435258 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:59.437150 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:59.437152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:59.437153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:59.437160 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:59.438247 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:59.438636 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:59.438637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:59.438637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:59.438697 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:59.438910 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:59.438910 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:59.438910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:59.439031 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:59.439080 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:59.439303 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 17:01:59.439367 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:01:59.439555 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:59.431898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:34597 (local address 127.30.194.3:37689)
0808 17:01:59.432034 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:59.432072 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:59.432519 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:59.432776 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:59.432779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:59.433304 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:59.433477 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:59.433481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.434119 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.434120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:59.434894 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:59.434896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.435271 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.435271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:59.435365 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:59.435766 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:59.435776 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:59.437056 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:59.438328 (+ 1272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:59.438334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:59.438336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:59.438559 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:59.438755 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:59.438756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:59.438756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:59.438865 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:59.439077 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:59.439082 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:59.439182 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 17:01:59.439279 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 17:01:59.439385 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260808 17:01:59.440028 5125 heartbeater.cc:347] Connected to a master server at 127.30.194.62:34597
I20260808 17:01:59.440121 5125 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:59.440297 5125 heartbeater.cc:511] Master 127.30.194.62:34597 requested a full tablet report, sending...
I20260808 17:01:59.440459 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 4880
I20260808 17:01:59.440558 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:01:59.440696 4332 ts_manager.cc:194] Registered new tserver with Master: d9e7ef3347ac4804b450213b8d5032a2 (127.30.194.3:43209)
I20260808 17:01:59.440769 31496 external_mini_cluster.cc:1428] Setting key 17ca5b4aa18f47eadb922bad41d87e21
I20260808 17:01:59.441116 4332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:37689
I20260808 17:01:59.441643 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[4253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208519, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[4253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208519, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:01:59.463936 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:59.457595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52566 (local address 127.30.194.62:34597)
0808 17:01:59.457645 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:59.457647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:59.457794 (+ 147us) server_negotiation.cc:407] Connection header received
0808 17:01:59.457856 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:59.457857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:59.457867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:59.457911 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:59.458418 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.458876 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.459562 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:59.459749 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:59.459801 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:59.461874 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:59.461875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:59.461876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:59.461882 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:59.462919 (+ 1037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:59.463211 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:59.463212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:59.463212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:59.463288 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:59.463446 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:59.463447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:59.463448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:59.463580 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:59.463667 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:59.463779 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 17:01:59.463851 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 17:01:59.464169 4332 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52566
I20260808 17:01:59.464576 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 4428
I20260808 17:01:59.490089 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 4669
I20260808 17:01:59.500567 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 4880
I20260808 17:01:59.513515 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 4280
2026-08-08T17:01:59Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1928 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[5145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[5145](info): set up 2 sockets
Aug 08 17:01:59 dist-test-slave-2j7r krb5kdc[5145](info): commencing operation
krb5kdc: starting...
W20260808 17:02:00.677186 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:02:00 dist-test-slave-2j7r krb5kdc[5145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:02:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:02:00.783412 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43297
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:43297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:00.866628 5167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:00.866880 5167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:00.866914 5167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:00.868530 5167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:00.868580 5167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:00.868598 5167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:00.868623 5167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:00.868645 5167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:00.870735 5167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:43297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:00.870919 5167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:00.871076 5167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:00.875523 5182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:00.875583 5185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:00.875530 5181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:00.875694 5167 server_base.cc:1034] running on GCE node
I20260808 17:02:00.875981 5167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:00.876264 5167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:00.877408 5167 hybrid_clock.cc:648] HybridClock initialized: now 1786208520877392 us; error 28 us; skew 500 ppm
Aug 08 17:02:00 dist-test-slave-2j7r krb5kdc[5145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208520, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:00.882362 5167 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:02:00.882752 5167 webserver.cc:460] Webserver started at http://127.30.194.62:45275/ using document root <none> and password file <none>
I20260808 17:02:00.882916 5167 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:00.882954 5167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:00.883050 5167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:00.883968 5167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "47b04db580d74305b54c533aa505847e"
format_stamp: "Formatted at 2026-08-08 17:02:00 on dist-test-slave-2j7r"
server_key: "bc1523bf11425a5e473f4d151e85d906"
server_key_iv: "8baa66404f43dc9b9e5413d978ab380b"
server_key_version: "encryptionkey@0"
I20260808 17:02:00.884256 5167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "47b04db580d74305b54c533aa505847e"
format_stamp: "Formatted at 2026-08-08 17:02:00 on dist-test-slave-2j7r"
server_key: "bc1523bf11425a5e473f4d151e85d906"
server_key_iv: "8baa66404f43dc9b9e5413d978ab380b"
server_key_version: "encryptionkey@0"
I20260808 17:02:00.885416 5167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:02:00.886134 5199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:00.886310 5167 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:02:00.886397 5167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "47b04db580d74305b54c533aa505847e"
format_stamp: "Formatted at 2026-08-08 17:02:00 on dist-test-slave-2j7r"
server_key: "bc1523bf11425a5e473f4d151e85d906"
server_key_iv: "8baa66404f43dc9b9e5413d978ab380b"
server_key_version: "encryptionkey@0"
I20260808 17:02:00.886464 5167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:00.913113 5167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:00.915060 5167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:00.915185 5167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:00.919153 5167 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:43297
I20260808 17:02:00.919425 5304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:43297 every 8 connection(s)
I20260808 17:02:00.919492 5167 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:00.920118 5306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:00.922452 5306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e: Bootstrap starting.
I20260808 17:02:00.923044 5306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:00.923332 5306 log.cc:824] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:00.923985 5306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e: No bootstrap required, opened a new log
I20260808 17:02:00.925578 5306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b04db580d74305b54c533aa505847e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43297 } }
I20260808 17:02:00.925726 5306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:00.925755 5306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b04db580d74305b54c533aa505847e, State: Initialized, Role: FOLLOWER
I20260808 17:02:00.925904 5306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b04db580d74305b54c533aa505847e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43297 } }
I20260808 17:02:00.925988 5306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:00.926024 5306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:00.926077 5306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:00.926895 5306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b04db580d74305b54c533aa505847e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43297 } }
I20260808 17:02:00.926990 5306 leader_election.cc:302] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47b04db580d74305b54c533aa505847e; no voters:
I20260808 17:02:00.927148 5306 leader_election.cc:288] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:00.927189 5310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:00.927333 5310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [term 1 LEADER]: Becoming Leader. State: Replica: 47b04db580d74305b54c533aa505847e, State: Running, Role: LEADER
I20260808 17:02:00.927398 5306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:00.927421 5310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b04db580d74305b54c533aa505847e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43297 } }
I20260808 17:02:00.927748 5311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47b04db580d74305b54c533aa505847e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b04db580d74305b54c533aa505847e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43297 } } }
I20260808 17:02:00.927764 5313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47b04db580d74305b54c533aa505847e. Latest consensus state: current_term: 1 leader_uuid: "47b04db580d74305b54c533aa505847e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b04db580d74305b54c533aa505847e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43297 } } }
I20260808 17:02:00.927856 5313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:00.927852 5311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:00.928011 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 5167
I20260808 17:02:00.928108 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:00.928145 5317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:00.928305 31496 external_mini_cluster.cc:1428] Setting key 963f09953b6870746d15673f34aff32c
I20260808 17:02:00.928623 5317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:00.930154 5317 catalog_manager.cc:1347] Generated new cluster ID: f7f1afb109be4b51aa4aaf1e448d8697
I20260808 17:02:00.930205 5317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:02:00 dist-test-slave-2j7r krb5kdc[5145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:00.936236 5317 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:00.936733 5317 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:00.940063 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:00.928815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57558 (local address 127.30.194.62:43297)
0808 17:02:00.929030 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:00.929039 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:00.929356 (+ 317us) server_negotiation.cc:407] Connection header received
0808 17:02:00.929840 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:00.929848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:00.929914 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:00.930028 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:00.930982 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:00.932330 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:00.933368 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:00.933956 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:00.934042 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:00.936699 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:00.936703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:00.936703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:00.936715 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:00.938531 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:00.938935 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:00.938938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:00.938939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:00.939030 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:00.939211 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:00.939211 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:00.939212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:00.939450 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:00.939583 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:00.939706 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 17:02:00.939837 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260808 17:02:00.940922 5317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47b04db580d74305b54c533aa505847e: Generated new TSK 0
I20260808 17:02:00.941051 5317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:02:00.967372 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43297
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:01.045635 5343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:01.045838 5343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:01.045871 5343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:01.047026 5343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:01.047066 5343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:01.047101 5343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:01.048569 5343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:01.048806 5343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:01.049001 5343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:02:01.054649 5343 server_base.cc:1034] running on GCE node
W20260808 17:02:01.054692 5353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:01.054632 5354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:01.054632 5358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:01.055013 5343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:01.055243 5343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:01.056409 5343 hybrid_clock.cc:648] HybridClock initialized: now 1786208521056396 us; error 31 us; skew 500 ppm
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[5145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208521, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:01.058789 5343 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:02:01.059180 5343 webserver.cc:460] Webserver started at http://127.30.194.1:45829/ using document root <none> and password file <none>
I20260808 17:02:01.059397 5343 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:01.059443 5343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:01.059564 5343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:01.060544 5343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "154745478d134f198ee11f9c209dc9cb"
format_stamp: "Formatted at 2026-08-08 17:02:01 on dist-test-slave-2j7r"
server_key: "511c435fc0a152ebebb356eca2d54b85"
server_key_iv: "d279aec557d7f6f312846a9d1accd1ec"
server_key_version: "encryptionkey@0"
I20260808 17:02:01.060860 5343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "154745478d134f198ee11f9c209dc9cb"
format_stamp: "Formatted at 2026-08-08 17:02:01 on dist-test-slave-2j7r"
server_key: "511c435fc0a152ebebb356eca2d54b85"
server_key_iv: "d279aec557d7f6f312846a9d1accd1ec"
server_key_version: "encryptionkey@0"
I20260808 17:02:01.062112 5343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:02:01.062788 5369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:01.062953 5343 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:01.063035 5343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "154745478d134f198ee11f9c209dc9cb"
format_stamp: "Formatted at 2026-08-08 17:02:01 on dist-test-slave-2j7r"
server_key: "511c435fc0a152ebebb356eca2d54b85"
server_key_iv: "d279aec557d7f6f312846a9d1accd1ec"
server_key_version: "encryptionkey@0"
I20260808 17:02:01.063097 5343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:01.084359 5343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:01.086102 5343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:01.086229 5343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:01.086427 5343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:01.086696 5343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:01.086730 5343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:01.086756 5343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:01.086774 5343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:01.092260 5343 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:41703
I20260808 17:02:01.092320 5571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:41703 every 8 connection(s)
I20260808 17:02:01.092595 5343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[5145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208521, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:01.100824 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:01.093281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:34223 (local address 127.30.194.62:43297)
0808 17:02:01.093342 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:01.093345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:01.093481 (+ 136us) server_negotiation.cc:407] Connection header received
0808 17:02:01.094395 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:01.094396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:01.094409 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:01.094459 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:01.095034 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.095529 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.096494 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.096702 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.096758 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:01.098620 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:01.098621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:01.098621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:01.098628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:01.099851 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:01.100150 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:01.100150 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:01.100151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:01.100195 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:01.100360 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:01.100361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:01.100361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:01.100450 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:01.100494 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:01.100680 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 17:02:01.100755 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:02:01.100965 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:01.093210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43297 (local address 127.30.194.1:34223)
0808 17:02:01.093372 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:01.093407 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:01.094275 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:01.094526 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:01.094529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:01.094761 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:01.094939 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:01.094943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.095610 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.095611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:01.096405 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:01.096408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.096771 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.096771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:01.096884 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:01.097297 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:01.097305 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:01.098536 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:01.099912 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:01.099915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:01.099916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:01.100100 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:01.100237 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:01.100237 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:01.100237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:01.100320 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:01.100489 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:01.100492 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:01.100592 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 17:02:01.100663 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:02:01.100769 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260808 17:02:01.101496 5573 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43297
I20260808 17:02:01.101576 5573 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:01.101708 5573 heartbeater.cc:511] Master 127.30.194.62:43297 requested a full tablet report, sending...
I20260808 17:02:01.101953 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 5343
I20260808 17:02:01.102046 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:02:01.102207 31496 external_mini_cluster.cc:1428] Setting key 7b366975ea8b78c1c1997cc688ff61af
I20260808 17:02:01.102214 5229 ts_manager.cc:194] Registered new tserver with Master: 154745478d134f198ee11f9c209dc9cb (127.30.194.1:41703)
I20260808 17:02:01.102849 5229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:34223
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:02:01.124711 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43297
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:01.204171 5590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:01.204356 5590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:01.204378 5590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:01.205490 5590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:01.205526 5590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:01.205551 5590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:01.206975 5590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:01.207152 5590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:01.207315 5590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:01.212044 5598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:01.212046 5599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:01.212095 5590 server_base.cc:1034] running on GCE node
W20260808 17:02:01.212046 5603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:01.212496 5590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:01.212702 5590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:01.213848 5590 hybrid_clock.cc:648] HybridClock initialized: now 1786208521213836 us; error 26 us; skew 500 ppm
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[5145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208521, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:01.216280 5590 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:02:01.216668 5590 webserver.cc:460] Webserver started at http://127.30.194.2:46813/ using document root <none> and password file <none>
I20260808 17:02:01.216840 5590 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:01.216879 5590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:01.216958 5590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:01.217926 5590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "a20de2b2c3f34bb1bf93e267aa3b308a"
format_stamp: "Formatted at 2026-08-08 17:02:01 on dist-test-slave-2j7r"
server_key: "f0c229ae7c3e5a368c1ff824d2ea6d8a"
server_key_iv: "bad1c79057c0cb3aab8b8636b11a8164"
server_key_version: "encryptionkey@0"
I20260808 17:02:01.218200 5590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "a20de2b2c3f34bb1bf93e267aa3b308a"
format_stamp: "Formatted at 2026-08-08 17:02:01 on dist-test-slave-2j7r"
server_key: "f0c229ae7c3e5a368c1ff824d2ea6d8a"
server_key_iv: "bad1c79057c0cb3aab8b8636b11a8164"
server_key_version: "encryptionkey@0"
I20260808 17:02:01.219412 5590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:01.220050 5620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:01.220206 5590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:01.220284 5590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "a20de2b2c3f34bb1bf93e267aa3b308a"
format_stamp: "Formatted at 2026-08-08 17:02:01 on dist-test-slave-2j7r"
server_key: "f0c229ae7c3e5a368c1ff824d2ea6d8a"
server_key_iv: "bad1c79057c0cb3aab8b8636b11a8164"
server_key_version: "encryptionkey@0"
I20260808 17:02:01.220347 5590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:01.246577 5590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:01.248255 5590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:01.248368 5590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:01.248574 5590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:01.248853 5590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:01.248885 5590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:01.248903 5590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:01.248916 5590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:01.254544 5590 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:35237
I20260808 17:02:01.254597 5829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:35237 every 8 connection(s)
I20260808 17:02:01.254863 5590 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:02:01.259037 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 5590
I20260808 17:02:01.259143 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:02:01.259322 31496 external_mini_cluster.cc:1428] Setting key dae803845614701ca635d20ef8c047a0
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[5145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208521, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:01.263501 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:01.255526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:36327 (local address 127.30.194.62:43297)
0808 17:02:01.255578 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:01.255580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:01.255803 (+ 223us) server_negotiation.cc:407] Connection header received
0808 17:02:01.256726 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:01.256727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:01.256738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:01.256796 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:01.257423 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.257918 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.258878 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.259070 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.259138 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:01.261067 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:01.261068 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:01.261069 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:01.261076 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:01.262289 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:01.262643 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:01.262644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:01.262644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:01.262698 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:01.262912 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:01.262913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:01.262913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:01.263021 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:01.263075 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:01.263331 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 17:02:01.263413 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:02:01.263643 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:01.255479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43297 (local address 127.30.194.2:36327)
0808 17:02:01.255651 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:01.255692 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:01.256617 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:01.256851 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:01.256854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:01.257127 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:01.257338 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:01.257343 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.257992 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.257993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:01.258781 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:01.258783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.259167 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.259169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:01.259311 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:01.259740 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:01.259748 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:01.260982 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:01.262363 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:01.262370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:01.262372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:01.262577 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:01.262758 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:01.262759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:01.262760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:01.262858 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:01.263074 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:01.263081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:01.263216 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 17:02:01.263300 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 17:02:01.263443 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260808 17:02:01.264336 5830 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43297
I20260808 17:02:01.264422 5830 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:01.264580 5830 heartbeater.cc:511] Master 127.30.194.62:43297 requested a full tablet report, sending...
I20260808 17:02:01.264990 5229 ts_manager.cc:194] Registered new tserver with Master: a20de2b2c3f34bb1bf93e267aa3b308a (127.30.194.2:35237)
I20260808 17:02:01.265486 5229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:36327
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:02:01.283154 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43297
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:01.363060 5839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:01.363246 5839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:01.363270 5839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:01.364373 5839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:01.364418 5839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:01.364449 5839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:01.365999 5839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:01.366204 5839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:01.366403 5839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:02:01.371125 5839 server_base.cc:1034] running on GCE node
W20260808 17:02:01.371084 5847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:01.371073 5851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:01.371097 5848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:01.371523 5839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:01.371752 5839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:01.372897 5839 hybrid_clock.cc:648] HybridClock initialized: now 1786208521372882 us; error 30 us; skew 500 ppm
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[5145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208521, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:01.375249 5839 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:02:01.375628 5839 webserver.cc:460] Webserver started at http://127.30.194.3:33367/ using document root <none> and password file <none>
I20260808 17:02:01.375816 5839 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:01.375864 5839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:01.375986 5839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:01.376919 5839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "c53de0879fef462eaca0aa5d51603892"
format_stamp: "Formatted at 2026-08-08 17:02:01 on dist-test-slave-2j7r"
server_key: "9383eb5a7bf65a3a15fa606130c48084"
server_key_iv: "8af5a73351d5bdad733274934fc11f83"
server_key_version: "encryptionkey@0"
I20260808 17:02:01.377295 5839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "c53de0879fef462eaca0aa5d51603892"
format_stamp: "Formatted at 2026-08-08 17:02:01 on dist-test-slave-2j7r"
server_key: "9383eb5a7bf65a3a15fa606130c48084"
server_key_iv: "8af5a73351d5bdad733274934fc11f83"
server_key_version: "encryptionkey@0"
I20260808 17:02:01.378504 5839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:01.379249 5864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:01.379400 5839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:01.379493 5839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "c53de0879fef462eaca0aa5d51603892"
format_stamp: "Formatted at 2026-08-08 17:02:01 on dist-test-slave-2j7r"
server_key: "9383eb5a7bf65a3a15fa606130c48084"
server_key_iv: "8af5a73351d5bdad733274934fc11f83"
server_key_version: "encryptionkey@0"
I20260808 17:02:01.379558 5839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:01.391928 5839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:01.393946 5839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:01.394069 5839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:01.394249 5839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:01.394517 5839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:01.394549 5839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:01.394574 5839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:01.394599 5839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:01.400157 5839 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:38089
I20260808 17:02:01.400216 6063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:38089 every 8 connection(s)
I20260808 17:02:01.400552 5839 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[5145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208521, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:01.407457 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 5839
I20260808 17:02:01.407567 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:02:01.407793 31496 external_mini_cluster.cc:1428] Setting key b9a9c17051dc70103fd04a4b1aeeaaae
I20260808 17:02:01.409646 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:01.401191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:48083 (local address 127.30.194.62:43297)
0808 17:02:01.401272 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:01.401274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:01.401484 (+ 210us) server_negotiation.cc:407] Connection header received
0808 17:02:01.402400 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:01.402402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:01.402415 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:01.402463 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:01.403070 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.403584 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.404528 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.404735 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.404789 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:01.406806 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:01.406808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:01.406808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:01.406817 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:01.408304 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:01.408773 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:01.408775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:01.408775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:01.408844 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:01.409058 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:01.409058 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:01.409059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:01.409238 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:01.409301 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:01.409514 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 17:02:01.409582 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 17:02:01.409762 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:01.401198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43297 (local address 127.30.194.3:48083)
0808 17:02:01.401376 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:01.401409 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:01.402277 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:01.402538 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:01.402542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:01.402797 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:01.402985 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:01.402988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.403663 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.403664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:01.404448 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:01.404450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.404802 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.404803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:01.404916 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:01.405360 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:01.405372 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:01.406698 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:01.408403 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:01.408410 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:01.408412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:01.408704 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:01.408907 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:01.408908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:01.408908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:01.409007 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:01.409296 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:01.409301 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:01.409409 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 17:02:01.409490 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:02:01.409600 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260808 17:02:01.410136 6064 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43297
I20260808 17:02:01.410235 6064 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:01.410411 6064 heartbeater.cc:511] Master 127.30.194.62:43297 requested a full tablet report, sending...
I20260808 17:02:01.410740 5229 ts_manager.cc:194] Registered new tserver with Master: c53de0879fef462eaca0aa5d51603892 (127.30.194.3:38089)
I20260808 17:02:01.411084 5229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:48083
I20260808 17:02:01.419572 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[5145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[5145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:02:01.443763 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:01.435824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.30.194.1:41703)
0808 17:02:01.435983 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:01.435989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:01.435998 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:01.436023 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:01.436025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:01.436040 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:01.436110 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:01.436721 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.437505 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.438286 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.438601 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.438668 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:01.440921 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:01.440930 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:01.440930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:01.440939 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:01.442521 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:01.442897 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:01.442901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:01.442902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:01.442978 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:01.443197 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:01.443199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:01.443199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:01.443394 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:01.443508 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:01.443603 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 17:02:01.443669 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[5145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:01.451318 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:01.445034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.30.194.62:43297)
0808 17:02:01.445111 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:01.445112 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:01.445121 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:01.445189 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:01.445190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:01.445230 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:01.445275 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:01.445725 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.446214 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.446857 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:01.447041 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:01.447093 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:01.449080 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:01.449082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:01.449082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:01.449112 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:01.450313 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:01.450626 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:01.450627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:01.450627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:01.450692 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:01.450851 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:01.450852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:01.450852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:01.451001 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:01.451077 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:01.451153 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 17:02:01.451245 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 17:02:01.451529 5229 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57562
I20260808 17:02:01.451859 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 5343
I20260808 17:02:01.462787 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 5590
I20260808 17:02:01.470732 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 5839
I20260808 17:02:01.508155 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 5167
2026-08-08T17:02:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1994 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[6088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[6088](info): set up 2 sockets
Aug 08 17:02:01 dist-test-slave-2j7r krb5kdc[6088](info): commencing operation
krb5kdc: starting...
W20260808 17:02:02.649132 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:02:02 dist-test-slave-2j7r krb5kdc[6088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:02:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:02:02.761174 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:45337
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:45337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:02.840893 6108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:02.841071 6108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:02.841113 6108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:02.842228 6108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:02.842268 6108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:02.842281 6108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:02.842291 6108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:02.842303 6108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:02.843752 6108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:45337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:45337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:02.843971 6108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:02.844167 6108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:02.848629 6124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:02.848630 6119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:02.848655 6117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:02.848814 6108 server_base.cc:1034] running on GCE node
I20260808 17:02:02.849056 6108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:02.849361 6108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:02.850502 6108 hybrid_clock.cc:648] HybridClock initialized: now 1786208522850487 us; error 29 us; skew 500 ppm
Aug 08 17:02:02 dist-test-slave-2j7r krb5kdc[6088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208522, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:02.852737 6108 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:02:02.853050 6108 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:02.853122 6108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:02.853230 6108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:02.854153 6108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "6d364608e2d04a218282437f0c2bb95f"
format_stamp: "Formatted at 2026-08-08 17:02:02 on dist-test-slave-2j7r"
server_key: "1d214b4cae4c78c06be201ec3dd48947"
server_key_iv: "215e7b2c8d80b2c5aae3de50453d5f01"
server_key_version: "encryptionkey@0"
I20260808 17:02:02.854451 6108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "6d364608e2d04a218282437f0c2bb95f"
format_stamp: "Formatted at 2026-08-08 17:02:02 on dist-test-slave-2j7r"
server_key: "1d214b4cae4c78c06be201ec3dd48947"
server_key_iv: "215e7b2c8d80b2c5aae3de50453d5f01"
server_key_version: "encryptionkey@0"
I20260808 17:02:02.855520 6108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:02:02.856089 6132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:02.856253 6108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:02.856324 6108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "6d364608e2d04a218282437f0c2bb95f"
format_stamp: "Formatted at 2026-08-08 17:02:02 on dist-test-slave-2j7r"
server_key: "1d214b4cae4c78c06be201ec3dd48947"
server_key_iv: "215e7b2c8d80b2c5aae3de50453d5f01"
server_key_version: "encryptionkey@0"
I20260808 17:02:02.856374 6108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:02.870297 6108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:02.872157 6108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:02.872269 6108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:02.875820 6108 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:45337
I20260808 17:02:02.875855 6229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:45337 every 8 connection(s)
I20260808 17:02:02.876073 6108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:02.876997 6232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:02.879083 6232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f: Bootstrap starting.
I20260808 17:02:02.879618 6232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:02.879877 6232 log.cc:824] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:02.880471 6232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f: No bootstrap required, opened a new log
I20260808 17:02:02.882014 6232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d364608e2d04a218282437f0c2bb95f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45337 } }
I20260808 17:02:02.882136 6232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:02.882170 6232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d364608e2d04a218282437f0c2bb95f, State: Initialized, Role: FOLLOWER
I20260808 17:02:02.882300 6232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d364608e2d04a218282437f0c2bb95f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45337 } }
I20260808 17:02:02.882375 6232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:02.882411 6232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:02.882454 6232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:02.883173 6232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d364608e2d04a218282437f0c2bb95f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45337 } }
I20260808 17:02:02.883260 6232 leader_election.cc:302] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d364608e2d04a218282437f0c2bb95f; no voters:
I20260808 17:02:02.883419 6232 leader_election.cc:288] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:02.883463 6236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:02.883597 6236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [term 1 LEADER]: Becoming Leader. State: Replica: 6d364608e2d04a218282437f0c2bb95f, State: Running, Role: LEADER
I20260808 17:02:02.883643 6232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:02.883687 6236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d364608e2d04a218282437f0c2bb95f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45337 } }
I20260808 17:02:02.884032 6240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d364608e2d04a218282437f0c2bb95f. Latest consensus state: current_term: 1 leader_uuid: "6d364608e2d04a218282437f0c2bb95f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d364608e2d04a218282437f0c2bb95f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45337 } } }
I20260808 17:02:02.884029 6239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d364608e2d04a218282437f0c2bb95f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d364608e2d04a218282437f0c2bb95f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45337 } } }
I20260808 17:02:02.884142 6240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:02.884148 6239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f [sys.catalog]: This master's current role is: LEADER
W20260808 17:02:02.885298 6274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:02:02.885340 6274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:02:02.885366 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 6108
I20260808 17:02:02.885385 6257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:02.885470 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:02.885543 6257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:02.885655 31496 external_mini_cluster.cc:1428] Setting key 370b6166846652ea41c82bc617fea36d
I20260808 17:02:02.887117 6257 catalog_manager.cc:1347] Generated new cluster ID: 9100a3b23bc04484a6fefbb0f4cdd81e
I20260808 17:02:02.887169 6257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:02:02 dist-test-slave-2j7r krb5kdc[6088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:02.893363 6257 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:02.893998 6257 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:02.896355 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:02.886079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34134 (local address 127.30.194.62:45337)
0808 17:02:02.886233 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:02.886242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:02.886283 (+ 41us) server_negotiation.cc:407] Connection header received
0808 17:02:02.886415 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:02.886422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:02.886490 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:02.886635 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:02.887355 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:02.888731 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:02.889795 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:02.890386 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:02.890451 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:02.893012 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:02.893016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:02.893017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:02.893038 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:02.894826 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:02.895183 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:02.895186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:02.895186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:02.895285 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:02.895458 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:02.895459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:02.895459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:02.895696 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:02.895793 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:02.896006 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 17:02:02.896135 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260808 17:02:02.897226 6257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f: Generated new TSK 0
I20260808 17:02:02.897357 6257 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:02:02.923883 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:45337
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:03.002401 6288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:03.002584 6288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:03.002606 6288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:03.003710 6288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:03.003747 6288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:03.003775 6288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:03.005299 6288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:03.005520 6288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:03.005715 6288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:03.010370 6301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.010392 6288 server_base.cc:1034] running on GCE node
W20260808 17:02:03.010347 6298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:03.010366 6296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.010712 6288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:03.010952 6288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:03.012089 6288 hybrid_clock.cc:648] HybridClock initialized: now 1786208523012077 us; error 29 us; skew 500 ppm
Aug 08 17:02:03 dist-test-slave-2j7r krb5kdc[6088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:03.014472 6288 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:02:03.014779 6288 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:03.014829 6288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:03.014945 6288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:03.015892 6288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "219bfa8a2d1940d1b987276c1b55b3e2"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "16f220e7010a2ae8549421f3acf98ccc"
server_key_iv: "c1bf13414dc6acde0b08e9e9c818b498"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.016196 6288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "219bfa8a2d1940d1b987276c1b55b3e2"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "16f220e7010a2ae8549421f3acf98ccc"
server_key_iv: "c1bf13414dc6acde0b08e9e9c818b498"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.017385 6288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:02:03.018069 6313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.018199 6288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:03.018304 6288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "219bfa8a2d1940d1b987276c1b55b3e2"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "16f220e7010a2ae8549421f3acf98ccc"
server_key_iv: "c1bf13414dc6acde0b08e9e9c818b498"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.018366 6288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:03.032493 6288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:03.034430 6288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:03.034551 6288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:03.034734 6288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:03.035000 6288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:03.035039 6288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.035058 6288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:03.035082 6288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.040331 6288 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:33897
I20260808 17:02:03.040390 6523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:33897 every 8 connection(s)
I20260808 17:02:03.040591 6288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:02:03 dist-test-slave-2j7r krb5kdc[6088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:03.048188 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 6288
I20260808 17:02:03.048285 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:02:03.048429 31496 external_mini_cluster.cc:1428] Setting key 3cd80acd2b2000c27ebe0bd986d3a6e6
I20260808 17:02:03.048816 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.041274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:34301 (local address 127.30.194.62:45337)
0808 17:02:03.041337 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.041340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.041526 (+ 186us) server_negotiation.cc:407] Connection header received
0808 17:02:03.041953 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.041954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.041966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.042017 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:03.042845 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.043333 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.044256 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.044459 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.044513 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.046425 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:03.046426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:03.046427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:03.046434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:03.047583 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:03.047925 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:03.047927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:03.047927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:03.047978 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:03.048179 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:03.048180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:03.048180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:03.048305 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:03.048377 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.048632 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.048723 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:02:03.048998 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.041263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:45337 (local address 127.30.194.1:34301)
0808 17:02:03.041421 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.041453 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.041864 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.042078 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.042082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.042307 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:03.042753 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.042757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.043412 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.043412 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.044178 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.044181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.044527 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.044528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.044642 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.044999 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:03.045006 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:03.046343 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:03.047654 (+ 1311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:03.047660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:03.047662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:03.047860 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:03.048032 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:03.048033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:03.048033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:03.048126 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:03.048366 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:03.048373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:03.048500 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.048596 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.048735 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260808 17:02:03.049467 6525 heartbeater.cc:347] Connected to a master server at 127.30.194.62:45337
I20260808 17:02:03.049546 6525 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:03.049675 6525 heartbeater.cc:511] Master 127.30.194.62:45337 requested a full tablet report, sending...
I20260808 17:02:03.050153 6166 ts_manager.cc:194] Registered new tserver with Master: 219bfa8a2d1940d1b987276c1b55b3e2 (127.30.194.1:33897)
I20260808 17:02:03.050844 6166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:34301
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:02:03.071811 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:45337
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:03.150887 6530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:03.151065 6530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:03.151086 6530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:03.152195 6530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:03.152235 6530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:03.152262 6530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:03.153766 6530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:03.153986 6530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:03.154176 6530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:03.159232 6545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.159322 6530 server_base.cc:1034] running on GCE node
W20260808 17:02:03.159416 6546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:03.159518 6550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.159684 6530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:03.159919 6530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:03.161067 6530 hybrid_clock.cc:648] HybridClock initialized: now 1786208523161054 us; error 30 us; skew 500 ppm
Aug 08 17:02:03 dist-test-slave-2j7r krb5kdc[6088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:03.163309 6530 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:02:03.163619 6530 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:03.163666 6530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:03.163791 6530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:03.164762 6530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "62ef4f0dcf0440088386f2f6b342bcbf"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "6e7f0820b70a7a2213ed319b77f804d3"
server_key_iv: "e05d2499a634eff46f355b7ddc2d7d4c"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.165071 6530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "62ef4f0dcf0440088386f2f6b342bcbf"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "6e7f0820b70a7a2213ed319b77f804d3"
server_key_iv: "e05d2499a634eff46f355b7ddc2d7d4c"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.166278 6530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:02:03.166944 6559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.167074 6530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:03.167151 6530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "62ef4f0dcf0440088386f2f6b342bcbf"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "6e7f0820b70a7a2213ed319b77f804d3"
server_key_iv: "e05d2499a634eff46f355b7ddc2d7d4c"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.167214 6530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:03.178809 6530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:03.180379 6530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:03.180490 6530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:03.180668 6530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:03.180943 6530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:03.180975 6530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.180995 6530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:03.181017 6530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.186726 6530 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:35999
I20260808 17:02:03.186774 6750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:35999 every 8 connection(s)
I20260808 17:02:03.186997 6530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:02:03 dist-test-slave-2j7r krb5kdc[6088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:03.195083 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.187598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:54271 (local address 127.30.194.62:45337)
0808 17:02:03.187645 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.187648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.187808 (+ 160us) server_negotiation.cc:407] Connection header received
0808 17:02:03.188245 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.188247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.188258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.188320 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:03.189135 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.189626 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.190548 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.190736 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.190787 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.192632 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:03.192634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:03.192634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:03.192642 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:03.193851 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:03.194207 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:03.194208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:03.194208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:03.194261 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:03.194446 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:03.194447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:03.194447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:03.194562 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:03.194626 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.194903 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.194990 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:02:03.195217 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.187546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:45337 (local address 127.30.194.2:54271)
0808 17:02:03.187699 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.187738 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.188139 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.188375 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.188378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.188603 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:03.189029 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.189033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.189702 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.189702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.190470 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.190472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.190793 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.190794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.190879 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.191227 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:03.191234 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:03.192540 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:03.193921 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:03.193927 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:03.193929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:03.194125 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:03.194311 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:03.194312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:03.194312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:03.194400 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:03.194633 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:03.194641 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:03.194774 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.194872 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.195018 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260808 17:02:03.195642 6753 heartbeater.cc:347] Connected to a master server at 127.30.194.62:45337
I20260808 17:02:03.195732 6753 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:03.195866 6753 heartbeater.cc:511] Master 127.30.194.62:45337 requested a full tablet report, sending...
I20260808 17:02:03.196211 6166 ts_manager.cc:194] Registered new tserver with Master: 62ef4f0dcf0440088386f2f6b342bcbf (127.30.194.2:35999)
I20260808 17:02:03.196496 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 6530
I20260808 17:02:03.196607 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:02:03.196669 6166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:54271
I20260808 17:02:03.196805 31496 external_mini_cluster.cc:1428] Setting key 4455220a9d20500839c71bb15dd22ef9
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:02:03.221915 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:45337
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:03.300644 6764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:03.300824 6764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:03.300845 6764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:03.301923 6764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:03.301954 6764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:03.301980 6764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:03.303339 6764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:03.303496 6764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:03.303653 6764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:03.308156 6776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:03.308272 6773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:03.308161 6774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.308359 6764 server_base.cc:1034] running on GCE node
I20260808 17:02:03.308589 6764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:03.308811 6764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:03.309952 6764 hybrid_clock.cc:648] HybridClock initialized: now 1786208523309937 us; error 27 us; skew 500 ppm
Aug 08 17:02:03 dist-test-slave-2j7r krb5kdc[6088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:03.312153 6764 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:02:03.312453 6764 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:03.312498 6764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:03.312613 6764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:03.313580 6764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "1a02b85a4a5946ea8e2c7ec7438c548e"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "32fc63f6e48695775bbfc27ab2daa274"
server_key_iv: "3289549ca81cd1c56f7973bb070d2c6b"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.313894 6764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "1a02b85a4a5946ea8e2c7ec7438c548e"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "32fc63f6e48695775bbfc27ab2daa274"
server_key_iv: "3289549ca81cd1c56f7973bb070d2c6b"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.315026 6764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:02:03.315675 6790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.315845 6764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:03.315919 6764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "1a02b85a4a5946ea8e2c7ec7438c548e"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "32fc63f6e48695775bbfc27ab2daa274"
server_key_iv: "3289549ca81cd1c56f7973bb070d2c6b"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.315987 6764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:03.331739 6764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:03.333674 6764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:03.333786 6764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:03.333950 6764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:03.334173 6764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:03.334200 6764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.334218 6764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:03.334231 6764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.339490 6764 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:38099
I20260808 17:02:03.339557 6967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:38099 every 8 connection(s)
I20260808 17:02:03.339769 6764 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:02:03 dist-test-slave-2j7r krb5kdc[6088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:03.346431 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 6764
I20260808 17:02:03.346563 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:02:03.346761 31496 external_mini_cluster.cc:1428] Setting key 18d649dcceacbf5d7195e85098f0885e
I20260808 17:02:03.348455 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.340417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:53017 (local address 127.30.194.62:45337)
0808 17:02:03.340462 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.340464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.340633 (+ 169us) server_negotiation.cc:407] Connection header received
0808 17:02:03.341120 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.341121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.341136 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.341198 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:03.342112 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.342612 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.343577 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.343786 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.343848 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.345943 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:03.345944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:03.345945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:03.345954 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:03.347315 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:03.347699 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:03.347700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:03.347700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:03.347769 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:03.347973 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:03.347973 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:03.347974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:03.348079 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:03.348137 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.348326 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.348393 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:02:03.348582 6972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.340377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:45337 (local address 127.30.194.3:53017)
0808 17:02:03.340511 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.340552 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.340984 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.341306 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.341310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.341549 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:03.342007 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.342011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.342702 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.342704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.343482 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.343484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.343862 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.343863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.343984 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.344433 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:03.344441 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:03.345833 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:03.347400 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:03.347406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:03.347408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:03.347630 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:03.347831 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:03.347831 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:03.347832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:03.347924 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:03.348128 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:03.348131 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:03.348231 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.348309 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.348414 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260808 17:02:03.348994 6968 heartbeater.cc:347] Connected to a master server at 127.30.194.62:45337
I20260808 17:02:03.349066 6968 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:03.349206 6968 heartbeater.cc:511] Master 127.30.194.62:45337 requested a full tablet report, sending...
I20260808 17:02:03.349563 6166 ts_manager.cc:194] Registered new tserver with Master: 1a02b85a4a5946ea8e2c7ec7438c548e (127.30.194.3:38099)
I20260808 17:02:03.349916 6166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:53017
I20260808 17:02:03.358455 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:02:03.364775 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.359642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.30.194.62:45337)
0808 17:02:03.359714 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.359716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.359844 (+ 128us) server_negotiation.cc:407] Connection header received
0808 17:02:03.359883 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.359884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.359895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.359966 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:03.360449 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.360905 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.361601 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.361790 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.361844 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.362788 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:03.362788 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:03.362789 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:03.362793 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:03.363947 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:03.364200 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:03.364201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:03.364201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:03.364245 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:03.364385 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:03.364386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:03.364386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:03.364502 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:03.364567 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.364638 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.364693 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:02:03.366063 6166 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:02:03.367522 6166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:02:03.373337 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.369578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33614 (local address 127.30.194.3:38099)
0808 17:02:03.369739 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.369743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.369862 (+ 119us) server_negotiation.cc:407] Connection header received
0808 17:02:03.369962 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.369965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.369980 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.370053 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:03.370252 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.371351 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.372422 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.373014 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.373114 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.373167 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.373232 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.373268 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":61,"threads_started":1}
I20260808 17:02:03.373337 6989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.369434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:38099 (local address 127.0.0.1:33614)
0808 17:02:03.369796 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.369807 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.369889 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.370122 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.370123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.370125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:03.370174 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.370175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.371465 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.371466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.372350 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.372352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.373118 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.373119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.373167 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.373169 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.373218 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.373258 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":35,"thread_start_us":78,"threads_started":1}
I20260808 17:02:03.373418 6994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.369653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34732 (local address 127.30.194.1:33897)
0808 17:02:03.369850 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.369853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.369862 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:03.369900 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.369902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.369916 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.369990 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:03.370174 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.370966 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.372229 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.372856 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.372916 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.372973 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.373305 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.373347 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":96,"threads_started":1}
I20260808 17:02:03.373719 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.369423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:35999 (local address 127.0.0.1:46656)
0808 17:02:03.369691 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.369741 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.369806 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.370138 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.370138 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.370140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:03.370172 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.370174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.371476 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.371477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.372559 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.372561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.373503 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.373504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.373570 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.373573 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.373617 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.373645 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":29,"thread_start_us":54,"threads_started":1}
I20260808 17:02:03.373783 6991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.369606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.30.194.2:35999)
0808 17:02:03.369927 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.369932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.369940 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:03.369974 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.369975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.369988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.370059 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:03.370260 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.371382 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.372638 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.373344 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.373408 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.373465 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.373655 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.373701 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":82,"threads_started":1}
I20260808 17:02:03.374040 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.369423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33897 (local address 127.0.0.1:34732)
0808 17:02:03.369762 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.369770 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.369808 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.369992 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.369995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.369997 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:03.370057 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.370059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.371048 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.371049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.372136 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.372138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.373025 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.373026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.373123 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.373139 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.373206 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.373258 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":3,"thread_start_us":52,"threads_started":1}
I20260808 17:02:03.374361 6867 tablet_service.cc:1467] Processing CreateTablet for tablet 3d69360be9544d768eb7fc689304a196 (DEFAULT_TABLE table=test-table [id=3304ae2a65c949a2acbc52becd4b6b9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:03.374361 6644 tablet_service.cc:1467] Processing CreateTablet for tablet 3d69360be9544d768eb7fc689304a196 (DEFAULT_TABLE table=test-table [id=3304ae2a65c949a2acbc52becd4b6b9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:03.374696 6644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d69360be9544d768eb7fc689304a196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:03.374693 6867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d69360be9544d768eb7fc689304a196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:03.376979 6995 tablet_bootstrap.cc:492] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: Bootstrap starting.
I20260808 17:02:03.377339 6996 tablet_bootstrap.cc:492] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: Bootstrap starting.
I20260808 17:02:03.377522 6407 tablet_service.cc:1467] Processing CreateTablet for tablet 3d69360be9544d768eb7fc689304a196 (DEFAULT_TABLE table=test-table [id=3304ae2a65c949a2acbc52becd4b6b9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:03.377782 6995 tablet_bootstrap.cc:654] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:03.377852 6407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d69360be9544d768eb7fc689304a196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:03.377944 6996 tablet_bootstrap.cc:654] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:03.378296 6995 log.cc:824] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:03.378494 6996 log.cc:824] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:03.378959 6995 tablet_bootstrap.cc:492] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: No bootstrap required, opened a new log
I20260808 17:02:03.379032 6995 ts_tablet_manager.cc:1397] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:02:03.379230 6996 tablet_bootstrap.cc:492] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: No bootstrap required, opened a new log
I20260808 17:02:03.379299 6996 ts_tablet_manager.cc:1397] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:03.380529 7002 tablet_bootstrap.cc:492] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2: Bootstrap starting.
I20260808 17:02:03.380939 6995 raft_consensus.cc:348] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 } } peers { permanent_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38099 } } peers { permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 } }
I20260808 17:02:03.381063 7002 tablet_bootstrap.cc:654] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:03.381122 6995 raft_consensus.cc:374] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:03.381165 6995 raft_consensus.cc:729] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a02b85a4a5946ea8e2c7ec7438c548e, State: Initialized, Role: FOLLOWER
I20260808 17:02:03.381210 6996 raft_consensus.cc:348] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 } } peers { permanent_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38099 } } peers { permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 } }
I20260808 17:02:03.381295 6995 consensus_queue.cc:260] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 } } peers { permanent_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38099 } } peers { permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 } }
I20260808 17:02:03.381352 6996 raft_consensus.cc:374] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:03.381392 6996 raft_consensus.cc:729] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62ef4f0dcf0440088386f2f6b342bcbf, State: Initialized, Role: FOLLOWER
I20260808 17:02:03.381497 6996 consensus_queue.cc:260] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 } } peers { permanent_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38099 } } peers { permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 } }
I20260808 17:02:03.381564 6995 ts_tablet_manager.cc:1428] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 17:02:03.381676 6968 heartbeater.cc:503] Master 127.30.194.62:45337 was elected leader, sending a full tablet report...
I20260808 17:02:03.381772 6996 ts_tablet_manager.cc:1428] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:02:03.381796 6753 heartbeater.cc:503] Master 127.30.194.62:45337 was elected leader, sending a full tablet report...
I20260808 17:02:03.381718 7002 log.cc:824] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:03.382810 7002 tablet_bootstrap.cc:492] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2: No bootstrap required, opened a new log
I20260808 17:02:03.382876 7002 ts_tablet_manager.cc:1397] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:03.384378 7002 raft_consensus.cc:348] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 } } peers { permanent_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38099 } } peers { permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 } }
I20260808 17:02:03.384491 7002 raft_consensus.cc:374] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:03.384522 7002 raft_consensus.cc:729] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 219bfa8a2d1940d1b987276c1b55b3e2, State: Initialized, Role: FOLLOWER
I20260808 17:02:03.384624 7002 consensus_queue.cc:260] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 } } peers { permanent_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38099 } } peers { permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 } }
I20260808 17:02:03.384858 7002 ts_tablet_manager.cc:1428] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:02:03.384902 6525 heartbeater.cc:503] Master 127.30.194.62:45337 was elected leader, sending a full tablet report...
I20260808 17:02:03.417557 7003 raft_consensus.cc:482] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:03.417655 7003 raft_consensus.cc:504] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 } } peers { permanent_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38099 } } peers { permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 } }
I20260808 17:02:03.417973 7003 leader_election.cc:288] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62ef4f0dcf0440088386f2f6b342bcbf (127.30.194.2:35999), 219bfa8a2d1940d1b987276c1b55b3e2 (127.30.194.1:33897)
I20260808 17:02:03.420940 6972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.418097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:35999 (local address 127.30.194.3:41435)
0808 17:02:03.418151 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.418161 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.418198 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.418359 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.418359 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.418361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:03.418416 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.418417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.418988 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.418989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.420082 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.420085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.420726 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.420727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.420800 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.420803 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.420854 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.420889 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 17:02:03.420979 6991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.418159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:41435 (local address 127.30.194.2:35999)
0808 17:02:03.418199 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.418201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.418212 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:02:03.418294 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.418295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.418304 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.418349 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:03.418469 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.418921 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.420178 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.420612 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.420700 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.420710 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.420873 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.420913 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:02:03.421058 6994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.418175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:42367 (local address 127.30.194.1:33897)
0808 17:02:03.418257 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.418259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.418366 (+ 107us) server_negotiation.cc:407] Connection header received
0808 17:02:03.418450 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.418451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.418480 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.418538 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:03.418677 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.419161 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.420364 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.420723 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.420791 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.420800 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.420967 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.421003 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 17:02:03.421072 7009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.418097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33897 (local address 127.30.194.3:42367)
0808 17:02:03.418295 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.418305 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.418385 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.418543 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.418544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.418545 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:03.418614 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.418616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.419246 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.419248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.420288 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:03.420290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.420804 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.420805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:03.420885 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.420889 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.420952 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.420990 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":11,"thread_start_us":116,"threads_started":1}
I20260808 17:02:03.421281 6683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d69360be9544d768eb7fc689304a196" candidate_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" is_pre_election: true
I20260808 17:02:03.421418 6683 raft_consensus.cc:2454] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a02b85a4a5946ea8e2c7ec7438c548e in term 0.
I20260808 17:02:03.421422 6445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d69360be9544d768eb7fc689304a196" candidate_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" is_pre_election: true
I20260808 17:02:03.421535 6445 raft_consensus.cc:2454] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a02b85a4a5946ea8e2c7ec7438c548e in term 0.
I20260808 17:02:03.421656 6793 leader_election.cc:302] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a02b85a4a5946ea8e2c7ec7438c548e, 62ef4f0dcf0440088386f2f6b342bcbf; no voters:
I20260808 17:02:03.421792 7003 raft_consensus.cc:2781] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:03.421842 7003 raft_consensus.cc:482] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:03.421875 7003 raft_consensus.cc:3037] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:03.422506 7003 raft_consensus.cc:504] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 } } peers { permanent_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38099 } } peers { permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 } }
I20260808 17:02:03.422621 7003 leader_election.cc:288] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [CANDIDATE]: Term 1 election: Requested vote from peers 62ef4f0dcf0440088386f2f6b342bcbf (127.30.194.2:35999), 219bfa8a2d1940d1b987276c1b55b3e2 (127.30.194.1:33897)
I20260808 17:02:03.422749 6445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d69360be9544d768eb7fc689304a196" candidate_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "219bfa8a2d1940d1b987276c1b55b3e2"
I20260808 17:02:03.422816 6445 raft_consensus.cc:3037] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:03.422797 6683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d69360be9544d768eb7fc689304a196" candidate_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62ef4f0dcf0440088386f2f6b342bcbf"
I20260808 17:02:03.422871 6683 raft_consensus.cc:3037] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:03.423439 6445 raft_consensus.cc:2454] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a02b85a4a5946ea8e2c7ec7438c548e in term 1.
I20260808 17:02:03.423588 6683 raft_consensus.cc:2454] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a02b85a4a5946ea8e2c7ec7438c548e in term 1.
I20260808 17:02:03.423616 6795 leader_election.cc:302] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a02b85a4a5946ea8e2c7ec7438c548e, 219bfa8a2d1940d1b987276c1b55b3e2; no voters:
I20260808 17:02:03.423688 7003 raft_consensus.cc:2781] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:03.423830 7003 raft_consensus.cc:686] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 1 LEADER]: Becoming Leader. State: Replica: 1a02b85a4a5946ea8e2c7ec7438c548e, State: Running, Role: LEADER
I20260808 17:02:03.423911 7003 consensus_queue.cc:237] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 } } peers { permanent_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38099 } } peers { permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 } }
I20260808 17:02:03.424711 6164 catalog_manager.cc:5495] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a02b85a4a5946ea8e2c7ec7438c548e (127.30.194.3). New cstate: current_term: 1 leader_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a02b85a4a5946ea8e2c7ec7438c548e" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 } health_report { overall_health: UNKNOWN } } }
I20260808 17:02:03.430948 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.428933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33616 (local address 127.30.194.3:38099)
0808 17:02:03.428973 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.428975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.428982 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:03.429035 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.429036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.429050 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.429112 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:03.429214 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.429629 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.430345 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:03.430528 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:03.430590 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:03.430715 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:03.430799 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:03.430857 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.430871 (+ 14us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.430887 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:02:03.432411 6683 raft_consensus.cc:1261] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf [term 1 FOLLOWER]: Refusing update from remote peer 1a02b85a4a5946ea8e2c7ec7438c548e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:03.432456 6445 raft_consensus.cc:1261] T 3d69360be9544d768eb7fc689304a196 P 219bfa8a2d1940d1b987276c1b55b3e2 [term 1 FOLLOWER]: Refusing update from remote peer 1a02b85a4a5946ea8e2c7ec7438c548e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:03.432720 7003 consensus_queue.cc:1035] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [LEADER]: Connected to new peer: Peer: permanent_uuid: "62ef4f0dcf0440088386f2f6b342bcbf" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:03.432857 7020 consensus_queue.cc:1035] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [LEADER]: Connected to new peer: Peer: permanent_uuid: "219bfa8a2d1940d1b987276c1b55b3e2" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:03.435041 7026 mvcc.cc:205] Tried to move back new op lower bound from 7316310111977091072 to 7316310111944966144. Current Snapshot: MvccSnapshot[applied={T|T < 7316310111977091072}]
I20260808 17:02:03.435912 7028 mvcc.cc:205] Tried to move back new op lower bound from 7316310111977091072 to 7316310111944966144. Current Snapshot: MvccSnapshot[applied={T|T < 7316310111977091072 or (T in {7316310111977091072})}]
I20260808 17:02:03.437726 6166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:03.437821 6166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:03.438463 6166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d364608e2d04a218282437f0c2bb95f: Sending DeleteTablet for 3 replicas of tablet 3d69360be9544d768eb7fc689304a196
I20260808 17:02:03.438758 6867 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d69360be9544d768eb7fc689304a196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:03 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:33614
I20260808 17:02:03.438786 6407 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d69360be9544d768eb7fc689304a196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:03 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:34732
I20260808 17:02:03.438802 6644 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d69360be9544d768eb7fc689304a196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:03 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:46656
I20260808 17:02:03.439080 7038 tablet_replica.cc:330] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: stopping tablet replica
I20260808 17:02:03.439178 7038 raft_consensus.cc:2229] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:02:03.439283 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 6288
I20260808 17:02:03.439302 7038 raft_consensus.cc:2258] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:03.450078 7039 tablet_replica.cc:330] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: stopping tablet replica
I20260808 17:02:03.450178 7038 ts_tablet_manager.cc:1905] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:03.450313 7039 raft_consensus.cc:2229] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:03.450457 7039 raft_consensus.cc:2258] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:03.451521 7039 ts_tablet_manager.cc:1905] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:03.452105 7038 ts_tablet_manager.cc:1918] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:03.452169 7038 log.cc:1196] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/3d69360be9544d768eb7fc689304a196
I20260808 17:02:03.452414 7038 ts_tablet_manager.cc:1939] T 3d69360be9544d768eb7fc689304a196 P 1a02b85a4a5946ea8e2c7ec7438c548e: Deleting consensus metadata
I20260808 17:02:03.452792 6135 catalog_manager.cc:4841] TS 1a02b85a4a5946ea8e2c7ec7438c548e (127.30.194.3:38099): tablet 3d69360be9544d768eb7fc689304a196 (table test-table [id=3304ae2a65c949a2acbc52becd4b6b9e]) successfully deleted
I20260808 17:02:03.453013 7039 ts_tablet_manager.cc:1918] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:03.453109 7039 log.cc:1196] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/3d69360be9544d768eb7fc689304a196
I20260808 17:02:03.453374 7039 ts_tablet_manager.cc:1939] T 3d69360be9544d768eb7fc689304a196 P 62ef4f0dcf0440088386f2f6b342bcbf: Deleting consensus metadata
I20260808 17:02:03.453792 6138 catalog_manager.cc:4841] TS 62ef4f0dcf0440088386f2f6b342bcbf (127.30.194.2:35999): tablet 3d69360be9544d768eb7fc689304a196 (table test-table [id=3304ae2a65c949a2acbc52becd4b6b9e]) successfully deleted
W20260808 17:02:03.456908 6143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:33897 (error 108)
I20260808 17:02:03.457276 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 6530
I20260808 17:02:03.457415 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.457278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33897 (local address 127.0.0.1:34736)
0808 17:02:03.457335 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.457380 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:33897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 17:02:03.462882 6143 catalog_manager.cc:4584] TS 219bfa8a2d1940d1b987276c1b55b3e2 (127.30.194.1:33897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d69360be9544d768eb7fc689304a196: Network error: Client connection negotiation failed: client connection to 127.30.194.1:33897: connect: Connection refused (error 111)
I20260808 17:02:03.467321 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 6764
I20260808 17:02:03.476197 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 6108
2026-08-08T17:02:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1969 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T17:02:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:03Z Disabled control of system clock
I20260808 17:02:03.508965 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:37073
--webserver_interface=127.30.194.62
--webserver_port=0
--builtin_ntp_servers=127.30.194.20:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:37073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 17:02:03.587745 7049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:03.587929 7049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:03.587947 7049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:03.589049 7049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:03.589100 7049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:03.589113 7049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:03.589124 7049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:03.589135 7049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:03.590441 7049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:37073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:37073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:03.590662 7049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:03.590931 7049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:03.595440 7061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.595534 7049 server_base.cc:1034] running on GCE node
W20260808 17:02:03.595446 7060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:03.595439 7066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.595820 7049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:03.596091 7049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:03.597223 7049 hybrid_clock.cc:648] HybridClock initialized: now 1786208523597201 us; error 33 us; skew 500 ppm
I20260808 17:02:03.598402 7049 webserver.cc:460] Webserver started at http://127.30.194.62:46267/ using document root <none> and password file <none>
I20260808 17:02:03.598647 7049 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:03.598701 7049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:03.598834 7049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:03.599869 7049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "2426f377936e474f84990391673745bf"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "99c3f29b442a721c6984fad415c42502"
server_key_iv: "da63c53d8fd65cbfdfc34d575a240ed0"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.600236 7049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "2426f377936e474f84990391673745bf"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "99c3f29b442a721c6984fad415c42502"
server_key_iv: "da63c53d8fd65cbfdfc34d575a240ed0"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.601552 7049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:02:03.602246 7074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.602415 7049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:03.602501 7049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "2426f377936e474f84990391673745bf"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "99c3f29b442a721c6984fad415c42502"
server_key_iv: "da63c53d8fd65cbfdfc34d575a240ed0"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.602576 7049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:03.605816 7049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:03.606040 7049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:03.606138 7049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:03.609969 7049 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:37073
I20260808 17:02:03.609999 7166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:37073 every 8 connection(s)
I20260808 17:02:03.610304 7049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:03.610899 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:03.612841 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 7049
I20260808 17:02:03.612936 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:03.613121 31496 external_mini_cluster.cc:1428] Setting key b3e9d8b16e00583643aed0fe3fee0f28
I20260808 17:02:03.613493 7170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf: Bootstrap starting.
I20260808 17:02:03.614061 7170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:03.614338 7170 log.cc:824] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:03.615060 7170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf: No bootstrap required, opened a new log
I20260808 17:02:03.615515 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.613605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.30.194.62:37073)
0808 17:02:03.613783 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.613789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.613814 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:02:03.614250 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.614257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.614297 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.614428 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:03.614981 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:03.614985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:03.614985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:03.614990 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:03.615036 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:03.615050 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:03.615241 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.615284 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.615338 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
I20260808 17:02:03.616606 7170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426f377936e474f84990391673745bf" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37073 } }
I20260808 17:02:03.616739 7170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:03.616765 7170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2426f377936e474f84990391673745bf, State: Initialized, Role: FOLLOWER
I20260808 17:02:03.616883 7170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426f377936e474f84990391673745bf" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37073 } }
I20260808 17:02:03.617285 7170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:03.617340 7170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:03.617396 7170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:03.617905 7170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426f377936e474f84990391673745bf" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37073 } }
I20260808 17:02:03.618003 7170 leader_election.cc:302] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2426f377936e474f84990391673745bf; no voters:
I20260808 17:02:03.618171 7170 leader_election.cc:288] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:03.618230 7182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:03.618362 7182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [term 1 LEADER]: Becoming Leader. State: Replica: 2426f377936e474f84990391673745bf, State: Running, Role: LEADER
I20260808 17:02:03.618431 7170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:03.618458 7182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426f377936e474f84990391673745bf" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37073 } }
I20260808 17:02:03.618804 7188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2426f377936e474f84990391673745bf. Latest consensus state: current_term: 1 leader_uuid: "2426f377936e474f84990391673745bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426f377936e474f84990391673745bf" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37073 } } }
I20260808 17:02:03.618809 7187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2426f377936e474f84990391673745bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2426f377936e474f84990391673745bf" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37073 } } }
I20260808 17:02:03.618942 7187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:03.619099 7188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:03.619519 7201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:03.619988 7201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:03.621392 7201 catalog_manager.cc:1347] Generated new cluster ID: 5d90b7a6830842fc910ff09c73f339b7
I20260808 17:02:03.621457 7201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:03.640228 7201 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:03.640367 7201 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:03.643277 7201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf: Generated new TSK 0
I20260808 17:02:03.643424 7201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:02:03.645313 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:37073
--builtin_ntp_servers=127.30.194.20:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 17:02:03.725524 7223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:03.725706 7223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:03.725726 7223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:03.726838 7223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:03.726877 7223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:03.726902 7223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:03.728174 7223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:03.728384 7223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:03.728642 7223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:03.734128 7236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:03.734134 7234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.734269 7223 server_base.cc:1034] running on GCE node
W20260808 17:02:03.734156 7233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.734578 7223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:03.734807 7223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:03.735955 7223 hybrid_clock.cc:648] HybridClock initialized: now 1786208523735937 us; error 33 us; skew 500 ppm
I20260808 17:02:03.737121 7223 webserver.cc:460] Webserver started at http://127.30.194.1:38819/ using document root <none> and password file <none>
I20260808 17:02:03.737339 7223 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:03.737389 7223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:03.737509 7223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:03.738481 7223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "b01a2f8f2f7b418691922aad20664e2b"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "6a21f0f7e95fc70b5178c62e7c7002c2"
server_key_iv: "c38f8456b3eefd8dd48aef7766de758d"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.738794 7223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "b01a2f8f2f7b418691922aad20664e2b"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "6a21f0f7e95fc70b5178c62e7c7002c2"
server_key_iv: "c38f8456b3eefd8dd48aef7766de758d"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.739960 7223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:02:03.740594 7246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.740725 7223 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:03.740810 7223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "b01a2f8f2f7b418691922aad20664e2b"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "6a21f0f7e95fc70b5178c62e7c7002c2"
server_key_iv: "c38f8456b3eefd8dd48aef7766de758d"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.740870 7223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:03.744076 7223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:03.744313 7223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:03.744429 7223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:03.744612 7223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:03.744885 7223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:03.744918 7223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.744943 7223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:03.744967 7223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.750329 7223 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:33887
I20260808 17:02:03.750361 7441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:33887 every 8 connection(s)
I20260808 17:02:03.750633 7223 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:02:03.753139 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.751208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:50369 (local address 127.30.194.62:37073)
0808 17:02:03.751291 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.751293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.751480 (+ 187us) server_negotiation.cc:407] Connection header received
0808 17:02:03.751881 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.751882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.751888 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.751934 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:03.752939 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:03.752940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:03.752940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:03.752943 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:03.752952 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:03.752956 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:03.752990 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.753048 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.753059 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 17:02:03.753293 7444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.751188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:37073 (local address 127.30.194.1:50369)
0808 17:02:03.751364 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.751397 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.751804 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.751987 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.751990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.751998 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:03.752822 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:03.752823 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:03.752851 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:03.752877 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:03.752994 (+ 117us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:03.752996 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:03.752998 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.753043 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.753103 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260808 17:02:03.753706 7442 heartbeater.cc:347] Connected to a master server at 127.30.194.62:37073
I20260808 17:02:03.753785 7442 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:03.753914 7442 heartbeater.cc:511] Master 127.30.194.62:37073 requested a full tablet report, sending...
I20260808 17:02:03.754360 7098 ts_manager.cc:194] Registered new tserver with Master: b01a2f8f2f7b418691922aad20664e2b (127.30.194.1:33887)
I20260808 17:02:03.760377 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 7223
I20260808 17:02:03.760459 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:02:03.760592 31496 external_mini_cluster.cc:1428] Setting key 400bdaddc375ed217b52ec04565a28e8
I20260808 17:02:03.761400 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:37073
--builtin_ntp_servers=127.30.194.20:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 17:02:03.844879 7445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:03.845070 7445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:03.845125 7445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:03.846264 7445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:03.846300 7445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:03.846334 7445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:03.847667 7445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:03.847880 7445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:03.848130 7445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:03.852808 7453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:03.852923 7451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.852970 7445 server_base.cc:1034] running on GCE node
W20260808 17:02:03.852835 7450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.853278 7445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:03.853518 7445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:03.854679 7445 hybrid_clock.cc:648] HybridClock initialized: now 1786208523854665 us; error 28 us; skew 500 ppm
I20260808 17:02:03.855878 7445 webserver.cc:460] Webserver started at http://127.30.194.2:33169/ using document root <none> and password file <none>
I20260808 17:02:03.856096 7445 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:03.856144 7445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:03.856274 7445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:03.857403 7445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "4037835101134ec3895a682b5e243af0"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "25c01bcd61d2dc8a18c482086198d67c"
server_key_iv: "24c3382a94cdc9ed3ff0928709f56abc"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.857728 7445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "4037835101134ec3895a682b5e243af0"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "25c01bcd61d2dc8a18c482086198d67c"
server_key_iv: "24c3382a94cdc9ed3ff0928709f56abc"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.858966 7445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:03.859640 7460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.859850 7445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:03.859935 7445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "4037835101134ec3895a682b5e243af0"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "25c01bcd61d2dc8a18c482086198d67c"
server_key_iv: "24c3382a94cdc9ed3ff0928709f56abc"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.859995 7445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:03.863329 7445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:03.863561 7445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:03.863679 7445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:03.863873 7445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:03.864154 7445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:03.864187 7445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.864212 7445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:03.864238 7445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.869599 7445 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:46101
I20260808 17:02:03.869652 7663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:46101 every 8 connection(s)
I20260808 17:02:03.869975 7445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:02:03.872763 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.870644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:39287 (local address 127.30.194.62:37073)
0808 17:02:03.870698 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.870700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.870907 (+ 207us) server_negotiation.cc:407] Connection header received
0808 17:02:03.871340 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.871341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.871349 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.871409 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:03.872533 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:03.872535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:03.872535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:03.872540 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:03.872559 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:03.872564 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:03.872626 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.872673 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.872687 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:02:03.872874 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.870616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:37073 (local address 127.30.194.2:39287)
0808 17:02:03.870799 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.870829 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.871253 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.871513 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.871516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.871525 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:03.872394 (+ 869us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:03.872394 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:03.872420 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:03.872447 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:03.872624 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:03.872627 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:03.872629 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.872675 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.872729 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260808 17:02:03.873409 7664 heartbeater.cc:347] Connected to a master server at 127.30.194.62:37073
I20260808 17:02:03.873485 7664 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:03.873622 7664 heartbeater.cc:511] Master 127.30.194.62:37073 requested a full tablet report, sending...
I20260808 17:02:03.873974 7098 ts_manager.cc:194] Registered new tserver with Master: 4037835101134ec3895a682b5e243af0 (127.30.194.2:46101)
I20260808 17:02:03.880113 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 7445
I20260808 17:02:03.880198 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:02:03.880390 31496 external_mini_cluster.cc:1428] Setting key 0fea31e74bf8f6a032eea8224bb2fc56
I20260808 17:02:03.881321 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:37073
--builtin_ntp_servers=127.30.194.20:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 17:02:03.972441 7667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:03.972620 7667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:03.972640 7667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:03.973742 7667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:03.973780 7667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:03.973814 7667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:03.975132 7667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:03.975330 7667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:03.975584 7667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:03.980232 7677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.980350 7667 server_base.cc:1034] running on GCE node
W20260808 17:02:03.980261 7676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:03.980274 7684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:03.980651 7667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:03.980880 7667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:03.982017 7667 hybrid_clock.cc:648] HybridClock initialized: now 1786208523982006 us; error 26 us; skew 500 ppm
I20260808 17:02:03.983100 7667 webserver.cc:460] Webserver started at http://127.30.194.3:46757/ using document root <none> and password file <none>
I20260808 17:02:03.983317 7667 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:03.983361 7667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:03.983484 7667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:03.984448 7667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "cd60ccebb9204810902c04a8253660de"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "0aebba97d278a2a70a6b533e863e0f3e"
server_key_iv: "0b2dfbe5521d33b8407d6c88b8a8faf1"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.984753 7667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "cd60ccebb9204810902c04a8253660de"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "0aebba97d278a2a70a6b533e863e0f3e"
server_key_iv: "0b2dfbe5521d33b8407d6c88b8a8faf1"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.985955 7667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:02:03.986579 7696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.986724 7667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:03.986814 7667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "cd60ccebb9204810902c04a8253660de"
format_stamp: "Formatted at 2026-08-08 17:02:03 on dist-test-slave-2j7r"
server_key: "0aebba97d278a2a70a6b533e863e0f3e"
server_key_iv: "0b2dfbe5521d33b8407d6c88b8a8faf1"
server_key_version: "encryptionkey@0"
I20260808 17:02:03.986868 7667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:03.990022 7667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:03.990227 7667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:03.990319 7667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:03.990468 7667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:03.990688 7667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:03.990716 7667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.990734 7667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:03.990751 7667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:03.996122 7667 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:36285
I20260808 17:02:03.996179 7882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:36285 every 8 connection(s)
I20260808 17:02:03.996445 7667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:02:03.996987 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 7667
I20260808 17:02:03.997121 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:02:03.997303 31496 external_mini_cluster.cc:1428] Setting key 20c190bdf852888d20417914ac142514
I20260808 17:02:03.999117 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.997077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:40425 (local address 127.30.194.62:37073)
0808 17:02:03.997142 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:03.997144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:03.997427 (+ 283us) server_negotiation.cc:407] Connection header received
0808 17:02:03.997865 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:03.997865 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:03.997874 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:03.997929 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:03.998937 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:03.998939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:03.998939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:03.998943 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:03.998958 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:03.998962 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:03.999001 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:03.999056 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 17:02:03.999067 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:02:03.999284 7886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:03.997079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:37073 (local address 127.30.194.3:40425)
0808 17:02:03.997310 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:03.997339 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:03.997772 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:03.997995 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:03.997999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:03.998008 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:03.998813 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:03.998813 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:03.998839 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:03.998880 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:03.999001 (+ 121us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:03.999003 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:03.999013 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 17:02:03.999061 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 17:02:03.999102 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":113,"threads_started":1}
I20260808 17:02:03.999682 7883 heartbeater.cc:347] Connected to a master server at 127.30.194.62:37073
I20260808 17:02:03.999755 7883 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:03.999892 7883 heartbeater.cc:511] Master 127.30.194.62:37073 requested a full tablet report, sending...
I20260808 17:02:04.000193 7098 ts_manager.cc:194] Registered new tserver with Master: cd60ccebb9204810902c04a8253660de (127.30.194.3:36285)
I20260808 17:02:04.008812 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:02:04.011066 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.010030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38226 (local address 127.30.194.62:37073)
0808 17:02:04.010102 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.010104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.010194 (+ 90us) server_negotiation.cc:407] Connection header received
0808 17:02:04.010287 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.010288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.010294 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.010341 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:04.010865 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:04.010866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:04.010867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:04.010871 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:04.010885 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:04.010889 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:04.010935 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:04.010985 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 17:02:04.010998 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 17:02:04.012528 7098 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:02:04.014025 7098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:02:04.017647 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.016222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.30.194.3:36285)
0808 17:02:04.016376 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.016380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.016389 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:04.016410 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.016412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.016420 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.016493 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:04.017318 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:04.017327 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:04.017327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:04.017332 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:04.017369 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:04.017392 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:04.017460 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:04.017561 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 17:02:04.017576 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260808 17:02:04.017678 7892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.015910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:46101 (local address 127.0.0.1:59570)
0808 17:02:04.016016 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:04.016053 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:04.016099 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:04.016519 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:04.016528 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:04.016535 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:04.017023 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:04.017024 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:04.017039 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:04.017060 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:04.017511 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:04.017513 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:04.017513 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 17:02:04.017574 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 17:02:04.017588 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":52,"threads_started":1}
I20260808 17:02:04.017724 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.016048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.30.194.2:46101)
0808 17:02:04.016377 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.016381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.016390 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:04.016411 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.016413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.016420 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.016495 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:04.017346 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:04.017354 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:04.017354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:04.017359 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:04.017391 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:04.017419 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:04.017513 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:04.017583 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 17:02:04.017599 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":98,"threads_started":1}
I20260808 17:02:04.017946 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.016039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:36285 (local address 127.0.0.1:33152)
0808 17:02:04.016208 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:04.016217 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:04.016257 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:04.016825 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:04.016826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:04.016830 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:04.017240 (+ 410us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:04.017241 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:04.017247 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:04.017258 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:04.017470 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:04.017478 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:04.017480 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:02:04.017547 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 17:02:04.017559 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260808 17:02:04.018030 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.016293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33887 (local address 127.0.0.1:41572)
0808 17:02:04.016658 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:04.016669 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:04.016767 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:04.016948 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:04.016949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:04.016952 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:04.017643 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:04.017643 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:04.017649 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:04.017658 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:04.017926 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:04.017927 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:04.017928 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 17:02:04.017964 (+ 36us) client_negotiation.cc:240] Negotiation successful
0808 17:02:04.017975 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":88,"threads_started":1}
I20260808 17:02:04.018245 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.016410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41572 (local address 127.30.194.1:33887)
0808 17:02:04.016600 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.016604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.016741 (+ 137us) server_negotiation.cc:407] Connection header received
0808 17:02:04.016835 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.016845 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.016857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.016933 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:04.017737 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:04.017744 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:04.017745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:04.017749 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:04.017830 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:04.017856 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:04.017904 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:04.018152 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 17:02:04.018168 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260808 17:02:04.018397 7545 tablet_service.cc:1467] Processing CreateTablet for tablet 83344a6da50e43b189c33556514d5f8e (DEFAULT_TABLE table=test-table [id=3374f9c437b94a5a8fa5599ff6c96e12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:04.018806 7545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83344a6da50e43b189c33556514d5f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:04.019057 7325 tablet_service.cc:1467] Processing CreateTablet for tablet 83344a6da50e43b189c33556514d5f8e (DEFAULT_TABLE table=test-table [id=3374f9c437b94a5a8fa5599ff6c96e12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:04.019055 7778 tablet_service.cc:1467] Processing CreateTablet for tablet 83344a6da50e43b189c33556514d5f8e (DEFAULT_TABLE table=test-table [id=3374f9c437b94a5a8fa5599ff6c96e12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:04.019380 7778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83344a6da50e43b189c33556514d5f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:04.019381 7325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83344a6da50e43b189c33556514d5f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:04.021481 7903 tablet_bootstrap.cc:492] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: Bootstrap starting.
I20260808 17:02:04.022106 7903 tablet_bootstrap.cc:654] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:04.022428 7903 log.cc:824] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:04.022584 7905 tablet_bootstrap.cc:492] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: Bootstrap starting.
I20260808 17:02:04.023080 7905 tablet_bootstrap.cc:654] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:04.023139 7903 tablet_bootstrap.cc:492] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: No bootstrap required, opened a new log
I20260808 17:02:04.023212 7903 ts_tablet_manager.cc:1397] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:04.023363 7905 log.cc:824] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:04.023635 7904 tablet_bootstrap.cc:492] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b: Bootstrap starting.
I20260808 17:02:04.024094 7905 tablet_bootstrap.cc:492] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: No bootstrap required, opened a new log
I20260808 17:02:04.024205 7905 ts_tablet_manager.cc:1397] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:04.024228 7904 tablet_bootstrap.cc:654] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:04.024531 7904 log.cc:824] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:04.025378 7903 raft_consensus.cc:348] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4037835101134ec3895a682b5e243af0" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 46101 } } peers { permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 } } peers { permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 } }
I20260808 17:02:04.025511 7903 raft_consensus.cc:374] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:04.025533 7903 raft_consensus.cc:729] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4037835101134ec3895a682b5e243af0, State: Initialized, Role: FOLLOWER
I20260808 17:02:04.025547 7904 tablet_bootstrap.cc:492] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b: No bootstrap required, opened a new log
I20260808 17:02:04.025601 7904 ts_tablet_manager.cc:1397] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:04.025624 7903 consensus_queue.cc:260] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4037835101134ec3895a682b5e243af0" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 46101 } } peers { permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 } } peers { permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 } }
I20260808 17:02:04.025887 7903 ts_tablet_manager.cc:1428] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:02:04.025915 7664 heartbeater.cc:503] Master 127.30.194.62:37073 was elected leader, sending a full tablet report...
I20260808 17:02:04.027166 7905 raft_consensus.cc:348] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4037835101134ec3895a682b5e243af0" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 46101 } } peers { permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 } } peers { permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 } }
I20260808 17:02:04.027323 7905 raft_consensus.cc:374] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:04.027370 7905 raft_consensus.cc:729] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd60ccebb9204810902c04a8253660de, State: Initialized, Role: FOLLOWER
I20260808 17:02:04.027487 7905 consensus_queue.cc:260] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4037835101134ec3895a682b5e243af0" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 46101 } } peers { permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 } } peers { permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 } }
I20260808 17:02:04.027752 7905 ts_tablet_manager.cc:1428] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:02:04.027906 7883 heartbeater.cc:503] Master 127.30.194.62:37073 was elected leader, sending a full tablet report...
I20260808 17:02:04.027974 7904 raft_consensus.cc:348] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4037835101134ec3895a682b5e243af0" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 46101 } } peers { permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 } } peers { permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 } }
I20260808 17:02:04.028116 7904 raft_consensus.cc:374] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:04.028159 7904 raft_consensus.cc:729] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b01a2f8f2f7b418691922aad20664e2b, State: Initialized, Role: FOLLOWER
I20260808 17:02:04.028270 7904 consensus_queue.cc:260] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4037835101134ec3895a682b5e243af0" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 46101 } } peers { permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 } } peers { permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 } }
I20260808 17:02:04.028502 7904 ts_tablet_manager.cc:1428] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:02:04.028546 7442 heartbeater.cc:503] Master 127.30.194.62:37073 was elected leader, sending a full tablet report...
W20260808 17:02:04.120497 7665 tablet.cc:2369] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:02:04.246942 7884 tablet.cc:2369] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:02:04.251240 7443 tablet.cc:2369] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:02:04.398649 7911 raft_consensus.cc:482] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:04.398767 7911 raft_consensus.cc:504] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4037835101134ec3895a682b5e243af0" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 46101 } } peers { permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 } } peers { permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 } }
I20260808 17:02:04.399086 7911 leader_election.cc:288] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd60ccebb9204810902c04a8253660de (127.30.194.3:36285), b01a2f8f2f7b418691922aad20664e2b (127.30.194.1:33887)
I20260808 17:02:04.400395 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.399354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:45047 (local address 127.30.194.3:36285)
0808 17:02:04.399410 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.399412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.399500 (+ 88us) server_negotiation.cc:407] Connection header received
0808 17:02:04.399567 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.399569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.399576 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.399635 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:04.400173 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:04.400174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:04.400175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:04.400179 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:04.400197 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:04.400201 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:04.400255 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:04.400309 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 17:02:04.400322 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:02:04.400408 7919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.399293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:36285 (local address 127.30.194.2:45047)
0808 17:02:04.399406 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:04.399418 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:04.399494 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:04.399658 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:04.399659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:04.399662 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:04.400085 (+ 423us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:04.400085 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:04.400092 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:04.400103 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:04.400256 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:04.400257 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:04.400258 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 17:02:04.400309 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:02:04.400322 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260808 17:02:04.400514 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.399544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:43101 (local address 127.30.194.1:33887)
0808 17:02:04.399590 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.399592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.399605 (+ 13us) server_negotiation.cc:407] Connection header received
0808 17:02:04.399663 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.399664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.399670 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.399736 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:04.400315 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:04.400316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:04.400317 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:04.400322 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:04.400340 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:04.400344 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:04.400391 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:04.400435 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 17:02:04.400449 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:02:04.400527 7920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.399452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33887 (local address 127.30.194.2:43101)
0808 17:02:04.399559 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:04.399568 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:04.399612 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:04.399739 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:04.399739 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:04.399741 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:04.400216 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:04.400217 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:04.400223 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:04.400234 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:04.400393 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:04.400394 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:04.400394 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 17:02:04.400436 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 17:02:04.400448 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260808 17:02:04.400885 7811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83344a6da50e43b189c33556514d5f8e" candidate_uuid: "4037835101134ec3895a682b5e243af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd60ccebb9204810902c04a8253660de" is_pre_election: true
I20260808 17:02:04.401008 7811 raft_consensus.cc:2454] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4037835101134ec3895a682b5e243af0 in term 0.
I20260808 17:02:04.401324 7463 leader_election.cc:302] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4037835101134ec3895a682b5e243af0, cd60ccebb9204810902c04a8253660de; no voters:
I20260808 17:02:04.401286 7357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83344a6da50e43b189c33556514d5f8e" candidate_uuid: "4037835101134ec3895a682b5e243af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b01a2f8f2f7b418691922aad20664e2b" is_pre_election: true
I20260808 17:02:04.401444 7357 raft_consensus.cc:2454] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4037835101134ec3895a682b5e243af0 in term 0.
I20260808 17:02:04.401458 7911 raft_consensus.cc:2781] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:04.401536 7911 raft_consensus.cc:482] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:04.401566 7911 raft_consensus.cc:3037] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:04.402289 7911 raft_consensus.cc:504] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4037835101134ec3895a682b5e243af0" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 46101 } } peers { permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 } } peers { permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 } }
I20260808 17:02:04.402406 7911 leader_election.cc:288] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [CANDIDATE]: Term 1 election: Requested vote from peers cd60ccebb9204810902c04a8253660de (127.30.194.3:36285), b01a2f8f2f7b418691922aad20664e2b (127.30.194.1:33887)
I20260808 17:02:04.402572 7811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83344a6da50e43b189c33556514d5f8e" candidate_uuid: "4037835101134ec3895a682b5e243af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd60ccebb9204810902c04a8253660de"
I20260808 17:02:04.402611 7357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83344a6da50e43b189c33556514d5f8e" candidate_uuid: "4037835101134ec3895a682b5e243af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b01a2f8f2f7b418691922aad20664e2b"
I20260808 17:02:04.402657 7811 raft_consensus.cc:3037] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:04.402675 7357 raft_consensus.cc:3037] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:04.403498 7811 raft_consensus.cc:2454] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4037835101134ec3895a682b5e243af0 in term 1.
I20260808 17:02:04.403498 7357 raft_consensus.cc:2454] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4037835101134ec3895a682b5e243af0 in term 1.
I20260808 17:02:04.403657 7463 leader_election.cc:302] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4037835101134ec3895a682b5e243af0, b01a2f8f2f7b418691922aad20664e2b; no voters:
I20260808 17:02:04.403738 7911 raft_consensus.cc:2781] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:04.403862 7911 raft_consensus.cc:686] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 1 LEADER]: Becoming Leader. State: Replica: 4037835101134ec3895a682b5e243af0, State: Running, Role: LEADER
I20260808 17:02:04.403939 7911 consensus_queue.cc:237] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4037835101134ec3895a682b5e243af0" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 46101 } } peers { permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 } } peers { permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 } }
I20260808 17:02:04.404708 7098 catalog_manager.cc:5495] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4037835101134ec3895a682b5e243af0 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "4037835101134ec3895a682b5e243af0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4037835101134ec3895a682b5e243af0" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 46101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 } health_report { overall_health: UNKNOWN } } }
I20260808 17:02:04.460146 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.459279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59584 (local address 127.30.194.2:46101)
0808 17:02:04.459342 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.459344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.459352 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:04.459382 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.459383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.459395 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.459441 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:04.459942 (+ 501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:04.459943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:04.459943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:04.459947 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:04.459961 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:04.459966 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:04.460014 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:04.460060 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 17:02:04.460074 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 17:02:04.461859 7811 raft_consensus.cc:1261] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de [term 1 FOLLOWER]: Refusing update from remote peer 4037835101134ec3895a682b5e243af0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:04.461880 7357 raft_consensus.cc:1261] T 83344a6da50e43b189c33556514d5f8e P b01a2f8f2f7b418691922aad20664e2b [term 1 FOLLOWER]: Refusing update from remote peer 4037835101134ec3895a682b5e243af0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:04.462167 7921 consensus_queue.cc:1035] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd60ccebb9204810902c04a8253660de" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 36285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:04.462301 7911 consensus_queue.cc:1035] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b01a2f8f2f7b418691922aad20664e2b" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:04.465181 7936 mvcc.cc:205] Tried to move back new op lower bound from 7316310116193329152 to 7316310115959148544. Current Snapshot: MvccSnapshot[applied={T|T < 7316310116193329152}]
I20260808 17:02:04.465276 7934 mvcc.cc:205] Tried to move back new op lower bound from 7316310116193329152 to 7316310115959148544. Current Snapshot: MvccSnapshot[applied={T|T < 7316310116193329152 or (T in {7316310116193329152})}]
I20260808 17:02:04.466320 7098 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:04.466413 7098 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:04.467121 7098 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2426f377936e474f84990391673745bf: Sending DeleteTablet for 3 replicas of tablet 83344a6da50e43b189c33556514d5f8e
I20260808 17:02:04.467458 7545 tablet_service.cc:1514] Processing DeleteTablet for tablet 83344a6da50e43b189c33556514d5f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:04 UTC) from {username='slave'} at 127.0.0.1:59570
I20260808 17:02:04.467463 7325 tablet_service.cc:1514] Processing DeleteTablet for tablet 83344a6da50e43b189c33556514d5f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:04 UTC) from {username='slave'} at 127.0.0.1:41572
I20260808 17:02:04.467684 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 7223
I20260808 17:02:04.467676 7778 tablet_service.cc:1514] Processing DeleteTablet for tablet 83344a6da50e43b189c33556514d5f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:04 UTC) from {username='slave'} at 127.0.0.1:33152
I20260808 17:02:04.468181 7949 tablet_replica.cc:330] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: stopping tablet replica
I20260808 17:02:04.468274 7949 raft_consensus.cc:2229] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:04.468365 7949 raft_consensus.cc:2258] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:04.470804 7949 ts_tablet_manager.cc:1905] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:04.472437 7949 ts_tablet_manager.cc:1918] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:04.472499 7949 log.cc:1196] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/83344a6da50e43b189c33556514d5f8e
I20260808 17:02:04.472754 7949 ts_tablet_manager.cc:1939] T 83344a6da50e43b189c33556514d5f8e P cd60ccebb9204810902c04a8253660de: Deleting consensus metadata
I20260808 17:02:04.473562 7946 tablet_replica.cc:330] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: stopping tablet replica
I20260808 17:02:04.473652 7946 raft_consensus.cc:2229] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:02:04.473752 7946 raft_consensus.cc:2258] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:04.475162 7076 catalog_manager.cc:4841] TS cd60ccebb9204810902c04a8253660de (127.30.194.3:36285): tablet 83344a6da50e43b189c33556514d5f8e (table test-table [id=3374f9c437b94a5a8fa5599ff6c96e12]) successfully deleted
I20260808 17:02:04.475421 7946 ts_tablet_manager.cc:1905] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:04.477208 7946 ts_tablet_manager.cc:1918] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:04.477283 7946 log.cc:1196] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/83344a6da50e43b189c33556514d5f8e
I20260808 17:02:04.477516 7946 ts_tablet_manager.cc:1939] T 83344a6da50e43b189c33556514d5f8e P 4037835101134ec3895a682b5e243af0: Deleting consensus metadata
I20260808 17:02:04.477910 7077 catalog_manager.cc:4841] TS 4037835101134ec3895a682b5e243af0 (127.30.194.2:46101): tablet 83344a6da50e43b189c33556514d5f8e (table test-table [id=3374f9c437b94a5a8fa5599ff6c96e12]) successfully deleted
W20260808 17:02:04.480638 7076 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:33887 (error 108)
I20260808 17:02:04.481133 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.480987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33887 (local address 127.0.0.1:41584)
0808 17:02:04.481041 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:04.481098 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:33887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:02:04.481169 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 7445
W20260808 17:02:04.487385 7076 catalog_manager.cc:4584] TS b01a2f8f2f7b418691922aad20664e2b (127.30.194.1:33887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83344a6da50e43b189c33556514d5f8e: Network error: Client connection negotiation failed: client connection to 127.30.194.1:33887: connect: Connection refused (error 111)
I20260808 17:02:04.494756 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 7667
I20260808 17:02:04.502631 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 7049
2026-08-08T17:02:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1021 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 17:02:04.523479 31496 mini_oidc.cc:152] Starting JWKS server
I20260808 17:02:04.524639 31496 webserver.cc:460] Webserver started at https://127.0.0.1:43567/ using document root <none> and password file <none>
I20260808 17:02:04.524745 31496 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 17:02:04.524906 31496 webserver.cc:460] Webserver started at http://127.0.0.1:46801/ using document root <none> and password file <none>
2026-08-08T17:02:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:04Z Disabled control of system clock
I20260808 17:02:04.532379 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:45377
--webserver_interface=127.30.194.62
--webserver_port=0
--builtin_ntp_servers=127.30.194.20:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:45377
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43567/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/testchainca.cert with env {}
W20260808 17:02:04.610455 7967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:04.610642 7967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:04.610673 7967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:04.611781 7967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:04.611817 7967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:04.611835 7967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:04.611850 7967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:04.611866 7967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 17:02:04.611888 7967 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 17:02:04.611912 7967 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43567/jwks.json
I20260808 17:02:04.613435 7967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:45377
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:45377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43567/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:04.613667 7967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:04.613852 7967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:04.618259 7973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:04.618206 7978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:04.618214 7974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:04.618471 7967 server_base.cc:1034] running on GCE node
I20260808 17:02:04.618690 7967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:04.618957 7967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:04.620086 7967 hybrid_clock.cc:648] HybridClock initialized: now 1786208524620071 us; error 29 us; skew 500 ppm
I20260808 17:02:04.621248 7967 webserver.cc:460] Webserver started at http://127.30.194.62:43203/ using document root <none> and password file <none>
I20260808 17:02:04.621459 7967 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:04.621505 7967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:04.621625 7967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:04.622560 7967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "a47fe102610b4a6bbdaf4af3d190132f"
format_stamp: "Formatted at 2026-08-08 17:02:04 on dist-test-slave-2j7r"
server_key: "a9ea52089be01f4c84cc8742ad525e55"
server_key_iv: "371bf61148ba5d18eb14c8d20e3ffcb3"
server_key_version: "encryptionkey@0"
I20260808 17:02:04.622881 7967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "a47fe102610b4a6bbdaf4af3d190132f"
format_stamp: "Formatted at 2026-08-08 17:02:04 on dist-test-slave-2j7r"
server_key: "a9ea52089be01f4c84cc8742ad525e55"
server_key_iv: "371bf61148ba5d18eb14c8d20e3ffcb3"
server_key_version: "encryptionkey@0"
I20260808 17:02:04.624006 7967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:02:04.624653 7987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:04.624792 7967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:04.624871 7967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "a47fe102610b4a6bbdaf4af3d190132f"
format_stamp: "Formatted at 2026-08-08 17:02:04 on dist-test-slave-2j7r"
server_key: "a9ea52089be01f4c84cc8742ad525e55"
server_key_iv: "371bf61148ba5d18eb14c8d20e3ffcb3"
server_key_version: "encryptionkey@0"
I20260808 17:02:04.624934 7967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:04.631942 7967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:04.632167 7967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:04.632278 7967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:04.635790 7967 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:45377
I20260808 17:02:04.635828 8070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:45377 every 8 connection(s)
I20260808 17:02:04.636101 7967 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:04.636224 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 7967
I20260808 17:02:04.636318 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:04.636521 31496 external_mini_cluster.cc:1428] Setting key 83c07822b1ca3566aee6ad688778747f
I20260808 17:02:04.637210 8071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:04.639721 8071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f: Bootstrap starting.
I20260808 17:02:04.640429 8071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:04.640734 8071 log.cc:824] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:04.641390 8071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f: No bootstrap required, opened a new log
I20260808 17:02:04.641810 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.636972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35552 (local address 127.30.194.62:45377)
0808 17:02:04.637170 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.637177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.637200 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:02:04.637596 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.637602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.637660 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.637771 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:04.638001 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.638812 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.639879 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.640445 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.640570 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:04.641131 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:04.641136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:04.641136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:04.641292 (+ 156us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:04.641322 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:04.641332 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:04.641385 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:04.641536 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 17:02:04.641617 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260808 17:02:04.642697 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:02:04.643359 8071 raft_consensus.cc:348] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47fe102610b4a6bbdaf4af3d190132f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45377 } }
I20260808 17:02:04.643504 8071 raft_consensus.cc:374] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:04.643538 8071 raft_consensus.cc:729] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a47fe102610b4a6bbdaf4af3d190132f, State: Initialized, Role: FOLLOWER
I20260808 17:02:04.643656 8071 consensus_queue.cc:260] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47fe102610b4a6bbdaf4af3d190132f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45377 } }
I20260808 17:02:04.643741 8071 raft_consensus.cc:388] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:04.643776 8071 raft_consensus.cc:482] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:04.643822 8071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:04.644304 8071 raft_consensus.cc:504] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47fe102610b4a6bbdaf4af3d190132f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45377 } }
I20260808 17:02:04.644392 8071 leader_election.cc:302] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a47fe102610b4a6bbdaf4af3d190132f; no voters:
I20260808 17:02:04.644548 8071 leader_election.cc:288] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:04.644599 8078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:04.644754 8078 raft_consensus.cc:686] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [term 1 LEADER]: Becoming Leader. State: Replica: a47fe102610b4a6bbdaf4af3d190132f, State: Running, Role: LEADER
I20260808 17:02:04.644788 8071 sys_catalog.cc:564] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:04.644845 8078 consensus_queue.cc:237] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47fe102610b4a6bbdaf4af3d190132f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45377 } }
I20260808 17:02:04.645186 8079 sys_catalog.cc:455] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a47fe102610b4a6bbdaf4af3d190132f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47fe102610b4a6bbdaf4af3d190132f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45377 } } }
I20260808 17:02:04.645200 8080 sys_catalog.cc:455] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a47fe102610b4a6bbdaf4af3d190132f. Latest consensus state: current_term: 1 leader_uuid: "a47fe102610b4a6bbdaf4af3d190132f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47fe102610b4a6bbdaf4af3d190132f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45377 } } }
I20260808 17:02:04.645309 8080 sys_catalog.cc:458] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:04.645300 8079 sys_catalog.cc:458] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:04.645570 8091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:04.646080 8091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:04.650017 8091 catalog_manager.cc:1347] Generated new cluster ID: 1013dd4667a14354ac28aa35450b7a54
I20260808 17:02:04.650059 8091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:04.680014 8091 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:04.680605 8091 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:04.683902 8091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a47fe102610b4a6bbdaf4af3d190132f: Generated new TSK 0
I20260808 17:02:04.684042 8091 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 17:02:04.718186 8115 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 17:02:04.718375 8115 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 17:02:04.718398 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.697127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35560 (local address 127.30.194.62:45377)
0808 17:02:04.697165 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.697167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.697309 (+ 142us) server_negotiation.cc:407] Connection header received
0808 17:02:04.697402 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.697403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.697414 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.697471 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0808 17:02:04.697603 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.698059 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.699140 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.699328 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.699374 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:04.699567 (+ 193us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 17:02:04.718152 (+ 18585us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 17:02:04.718337 (+ 185us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35560: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:02:04.718616 8074 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35560: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 17:02:04.718765 8114 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:04.697060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:45377 (local address 127.0.0.1:35560)
0808 17:02:04.697245 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:04.697255 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:04.697321 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:04.697492 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:04.697493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:04.697494 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 17:02:04.697550 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:04.697552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.698141 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.698143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:04.699069 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:04.699072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.699396 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.699397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:04.699458 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:04.699479 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 17:02:04.718277 (+ 18798us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 17:02:04.718378 (+ 101us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.194.62:45377: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.194.62:45377: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 17:02:04.719147 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 7967
2026-08-08T17:02:04Z chronyd exiting
I20260808 17:02:04.765515 31496 test_util.cc:175] -----------------------------------------------
I20260808 17:02:04.765583 31496 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208501350379-31496-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (243 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 17:02:04.766337 31496 mini_oidc.cc:152] Starting JWKS server
I20260808 17:02:04.766974 31496 webserver.cc:460] Webserver started at https://127.0.0.1:46471/ using document root <none> and password file <none>
I20260808 17:02:04.767017 31496 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 17:02:04.767155 31496 webserver.cc:460] Webserver started at http://127.0.0.1:34563/ using document root <none> and password file <none>
2026-08-08T17:02:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:04Z Disabled control of system clock
I20260808 17:02:04.774907 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:37367
--webserver_interface=127.30.194.62
--webserver_port=0
--builtin_ntp_servers=127.30.194.20:38415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:37367
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46471/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/testchainca.cert with env {}
W20260808 17:02:04.852994 8122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:04.853206 8122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:04.853240 8122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:04.854363 8122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:04.854403 8122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:04.854419 8122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:04.854434 8122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:04.854449 8122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 17:02:04.854466 8122 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 17:02:04.854485 8122 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46471/jwks.json
I20260808 17:02:04.855945 8122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:38415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:37367
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:37367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46471/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:04.856181 8122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:04.856381 8122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:04.860766 8137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:04.860899 8122 server_base.cc:1034] running on GCE node
W20260808 17:02:04.860793 8133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:04.860766 8135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:04.861254 8122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:04.861578 8122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:04.862727 8122 hybrid_clock.cc:648] HybridClock initialized: now 1786208524862702 us; error 42 us; skew 500 ppm
I20260808 17:02:04.863847 8122 webserver.cc:460] Webserver started at http://127.30.194.62:33965/ using document root <none> and password file <none>
I20260808 17:02:04.864087 8122 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:04.864148 8122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:04.864307 8122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:04.865545 8122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "19ffb8488a8644f28f6f9334dd6f7401"
format_stamp: "Formatted at 2026-08-08 17:02:04 on dist-test-slave-2j7r"
server_key: "7c5504396d1ef1e75cf969a79df1caf1"
server_key_iv: "8b4831717e124e0a280bccbf68836681"
server_key_version: "encryptionkey@0"
I20260808 17:02:04.865944 8122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "19ffb8488a8644f28f6f9334dd6f7401"
format_stamp: "Formatted at 2026-08-08 17:02:04 on dist-test-slave-2j7r"
server_key: "7c5504396d1ef1e75cf969a79df1caf1"
server_key_iv: "8b4831717e124e0a280bccbf68836681"
server_key_version: "encryptionkey@0"
I20260808 17:02:04.867725 8122 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 17:02:04.868558 8151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:04.868743 8122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:04.868841 8122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "19ffb8488a8644f28f6f9334dd6f7401"
format_stamp: "Formatted at 2026-08-08 17:02:04 on dist-test-slave-2j7r"
server_key: "7c5504396d1ef1e75cf969a79df1caf1"
server_key_iv: "8b4831717e124e0a280bccbf68836681"
server_key_version: "encryptionkey@0"
I20260808 17:02:04.868912 8122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:04.881670 8122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:04.881893 8122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:04.881982 8122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:04.885531 8122 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:37367
I20260808 17:02:04.885558 8246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:37367 every 8 connection(s)
I20260808 17:02:04.885834 8122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:04.886499 8247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:04.888784 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 8122
I20260808 17:02:04.888875 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:04.889055 31496 external_mini_cluster.cc:1428] Setting key 567f2e134734dbcd76d3438db7dbe0db
I20260808 17:02:04.889249 8247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401: Bootstrap starting.
I20260808 17:02:04.889971 8247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:04.890309 8247 log.cc:824] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:04.891083 8247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401: No bootstrap required, opened a new log
I20260808 17:02:04.893342 8247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ffb8488a8644f28f6f9334dd6f7401" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37367 } }
I20260808 17:02:04.893505 8247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:04.893551 8247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19ffb8488a8644f28f6f9334dd6f7401, State: Initialized, Role: FOLLOWER
I20260808 17:02:04.893694 8247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ffb8488a8644f28f6f9334dd6f7401" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37367 } }
I20260808 17:02:04.893783 8247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:04.893818 8247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:04.893869 8247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:04.894557 8247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ffb8488a8644f28f6f9334dd6f7401" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37367 } }
I20260808 17:02:04.894668 8247 leader_election.cc:302] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19ffb8488a8644f28f6f9334dd6f7401; no voters:
I20260808 17:02:04.894838 8247 leader_election.cc:288] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:04.894866 8254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:04.895004 8254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [term 1 LEADER]: Becoming Leader. State: Replica: 19ffb8488a8644f28f6f9334dd6f7401, State: Running, Role: LEADER
I20260808 17:02:04.895085 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.889517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.30.194.62:37367)
0808 17:02:04.889722 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.889730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.889768 (+ 38us) server_negotiation.cc:407] Connection header received
0808 17:02:04.890371 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.890380 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.890457 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.890606 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:04.890809 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.892169 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.892923 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.893554 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.893680 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:04.894283 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:04.894287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:04.894287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:04.894494 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:04.894537 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:04.894552 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:04.894632 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:04.894673 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 17:02:04.894859 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260808 17:02:04.895089 8254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ffb8488a8644f28f6f9334dd6f7401" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37367 } }
I20260808 17:02:04.895092 8247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:04.895433 8255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19ffb8488a8644f28f6f9334dd6f7401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ffb8488a8644f28f6f9334dd6f7401" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37367 } } }
I20260808 17:02:04.895550 8255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:04.896004 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:02:04.896808 8256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19ffb8488a8644f28f6f9334dd6f7401. Latest consensus state: current_term: 1 leader_uuid: "19ffb8488a8644f28f6f9334dd6f7401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ffb8488a8644f28f6f9334dd6f7401" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37367 } } }
I20260808 17:02:04.896894 8256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401 [sys.catalog]: This master's current role is: LEADER
W20260808 17:02:04.897073 8277 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:02:04.897146 8277 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:02:04.897207 8265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:04.897387 8265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:04.898800 8265 catalog_manager.cc:1347] Generated new cluster ID: 3afa094221ed4c5d9bb50c57a2102421
I20260808 17:02:04.898851 8265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:04.912642 8265 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:04.913105 8265 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:04.917783 8265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19ffb8488a8644f28f6f9334dd6f7401: Generated new TSK 0
I20260808 17:02:04.917950 8265 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 17:02:04.938326 8294 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 17:02:04.938380 8294 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 17:02:04.938707 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:04.924823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53002 (local address 127.30.194.62:37367)
0808 17:02:04.924867 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:04.924869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:04.924943 (+ 74us) server_negotiation.cc:407] Connection header received
0808 17:02:04.924977 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:04.924978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:04.924991 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:04.925079 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0808 17:02:04.925212 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.925958 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.926699 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.926998 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.927072 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:04.927212 (+ 140us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 17:02:04.938504 (+ 11292us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46471/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 17:02:04.938643 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53002: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46471/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 17:02:04.938762 8251 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53002: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46471/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 17:02:04.938766 8289 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:04.924761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:37367 (local address 127.0.0.1:53002)
0808 17:02:04.924876 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:04.924886 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:04.924935 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:04.925076 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:04.925076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:04.925078 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 17:02:04.925148 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:04.925150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.926035 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.926036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:04.926622 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:04.926623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:04.927078 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:04.927079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:04.927120 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:04.927125 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 17:02:04.938608 (+ 11483us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46471/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 17:02:04.938696 (+ 88us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.194.62:37367: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46471/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":71,"threads_started":1}
I20260808 17:02:04.939049 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 8122
2026-08-08T17:02:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 17:02:04.986223 31496 mini_oidc.cc:152] Starting JWKS server
I20260808 17:02:04.986850 31496 webserver.cc:460] Webserver started at https://127.0.0.1:46369/ using document root <none> and password file <none>
I20260808 17:02:04.986889 31496 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 17:02:04.987022 31496 webserver.cc:460] Webserver started at http://127.0.0.1:42271/ using document root <none> and password file <none>
2026-08-08T17:02:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:04Z Disabled control of system clock
I20260808 17:02:04.994643 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:41941
--webserver_interface=127.30.194.62
--webserver_port=0
--builtin_ntp_servers=127.30.194.20:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:41941
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46369/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 17:02:05.072942 8303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:05.073165 8303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:05.073200 8303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:05.074311 8303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:05.074349 8303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:05.074368 8303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:05.074383 8303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:05.074397 8303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 17:02:05.074414 8303 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 17:02:05.074441 8303 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46369/jwks.json
I20260808 17:02:05.075753 8303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:41941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:41941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46369/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:05.075984 8303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:05.076244 8303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:05.080825 8322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:05.080884 8315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:05.080814 8313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:05.080972 8303 server_base.cc:1034] running on GCE node
I20260808 17:02:05.081223 8303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:05.081511 8303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:05.082666 8303 hybrid_clock.cc:648] HybridClock initialized: now 1786208525082648 us; error 30 us; skew 500 ppm
I20260808 17:02:05.083820 8303 webserver.cc:460] Webserver started at http://127.30.194.62:33015/ using document root <none> and password file <none>
I20260808 17:02:05.084019 8303 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:05.084067 8303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:05.084192 8303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:05.085259 8303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3"
format_stamp: "Formatted at 2026-08-08 17:02:05 on dist-test-slave-2j7r"
server_key: "234187809b01f8af3aaf84e3409b6cc1"
server_key_iv: "573958889463284200597919d187eb6e"
server_key_version: "encryptionkey@0"
I20260808 17:02:05.085579 8303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3"
format_stamp: "Formatted at 2026-08-08 17:02:05 on dist-test-slave-2j7r"
server_key: "234187809b01f8af3aaf84e3409b6cc1"
server_key_iv: "573958889463284200597919d187eb6e"
server_key_version: "encryptionkey@0"
I20260808 17:02:05.086773 8303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:05.087407 8331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:05.087566 8303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:05.087661 8303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3"
format_stamp: "Formatted at 2026-08-08 17:02:05 on dist-test-slave-2j7r"
server_key: "234187809b01f8af3aaf84e3409b6cc1"
server_key_iv: "573958889463284200597919d187eb6e"
server_key_version: "encryptionkey@0"
I20260808 17:02:05.087723 8303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:05.118791 8303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:05.119055 8303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:05.119180 8303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:05.122843 8303 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:41941
I20260808 17:02:05.122874 8421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:41941 every 8 connection(s)
I20260808 17:02:05.123172 8303 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:05.123786 8422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:05.126339 8422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3: Bootstrap starting.
I20260808 17:02:05.126878 8422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:05.127121 8422 log.cc:824] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:05.127753 8422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3: No bootstrap required, opened a new log
I20260808 17:02:05.128602 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 8303
I20260808 17:02:05.128701 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:05.128868 31496 external_mini_cluster.cc:1428] Setting key 096badaab12bd2851085aec96ab146eb
I20260808 17:02:05.129590 8422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41941 } }
I20260808 17:02:05.129750 8422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:05.129783 8422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3189ed3ccb4da5937d2eb4ee069eb3, State: Initialized, Role: FOLLOWER
I20260808 17:02:05.129901 8422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41941 } }
I20260808 17:02:05.129978 8422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:05.130007 8422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:05.130049 8422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:05.130535 8422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41941 } }
I20260808 17:02:05.130628 8422 leader_election.cc:302] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f3189ed3ccb4da5937d2eb4ee069eb3; no voters:
I20260808 17:02:05.130779 8422 leader_election.cc:288] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:05.130820 8435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:05.130954 8435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 0f3189ed3ccb4da5937d2eb4ee069eb3, State: Running, Role: LEADER
I20260808 17:02:05.131013 8422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:05.131048 8435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41941 } }
I20260808 17:02:05.131350 8435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f3189ed3ccb4da5937d2eb4ee069eb3. Latest consensus state: current_term: 1 leader_uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41941 } } }
I20260808 17:02:05.131409 8435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:05.131536 8436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3189ed3ccb4da5937d2eb4ee069eb3" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 41941 } } }
I20260808 17:02:05.131589 8436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:05.131695 8442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:05.132197 8442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:05.133962 8442 catalog_manager.cc:1347] Generated new cluster ID: ff32c90b83ec48488ac2af6b8f5149ac
I20260808 17:02:05.134016 8442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:05.134577 8432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:05.129342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60182 (local address 127.30.194.62:41941)
0808 17:02:05.129564 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:05.129573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:05.129605 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:02:05.129708 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:05.129715 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:05.129792 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:05.129929 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:05.130123 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.131581 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.132615 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.133217 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.133327 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:05.133927 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:05.133931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:05.133931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:05.134119 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:05.134158 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:05.134173 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:05.134246 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:05.134318 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 17:02:05.134388 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260808 17:02:05.135372 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:02:05.141276 8442 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:05.141741 8442 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:05.152148 8442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f3189ed3ccb4da5937d2eb4ee069eb3: Generated new TSK 0
I20260808 17:02:05.152279 8442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:02:05.160740 8432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:05.157545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60194 (local address 127.30.194.62:41941)
0808 17:02:05.157591 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:05.157593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:05.157693 (+ 100us) server_negotiation.cc:407] Connection header received
0808 17:02:05.157762 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:05.157763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:05.157773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:05.157825 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0808 17:02:05.157940 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.158422 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.159482 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.159671 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.159714 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:05.159869 (+ 155us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 17:02:05.160560 (+ 691us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46369/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 17:02:05.160673 (+ 113us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60194: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46369/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 17:02:05.160794 8432 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60194: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46369/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 17:02:05.160840 8480 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:05.157488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:41941 (local address 127.0.0.1:60194)
0808 17:02:05.157625 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:05.157636 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:05.157701 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:05.157826 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:05.157827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:05.157829 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 17:02:05.157894 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:05.157896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.158488 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.158489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:05.159404 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:05.159406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.159725 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.159726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:05.159793 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:05.159798 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 17:02:05.160646 (+ 848us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46369/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 17:02:05.160744 (+ 98us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.194.62:41941: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46369/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260808 17:02:05.160955 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 8303
E20260808 17:02:05.183419 8482 webserver.cc:533] Webserver: sslize failed:
E20260808 17:02:05.183485 8482 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T17:02:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (224 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T17:02:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:05Z Disabled control of system clock
I20260808 17:02:05.218472 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:39087
--webserver_interface=127.30.194.62
--webserver_port=0
--builtin_ntp_servers=127.30.194.20:36799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:39087
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/testchainca.cert with env {}
W20260808 17:02:05.296442 8487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:05.296653 8487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:05.296674 8487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:05.297864 8487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:05.297906 8487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:05.297919 8487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:05.297930 8487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:05.297940 8487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 17:02:05.297952 8487 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 17:02:05.297976 8487 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 17:02:05.299420 8487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:39087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:39087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:05.299633 8487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:05.299829 8487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:05.304283 8494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:05.304327 8502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:05.304360 8496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:05.304879 8487 server_base.cc:1034] running on GCE node
I20260808 17:02:05.305143 8487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:05.305456 8487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:05.306608 8487 hybrid_clock.cc:648] HybridClock initialized: now 1786208525306582 us; error 37 us; skew 500 ppm
I20260808 17:02:05.307984 8487 webserver.cc:460] Webserver started at http://127.30.194.62:46763/ using document root <none> and password file <none>
I20260808 17:02:05.308269 8487 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:05.308328 8487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:05.308459 8487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:05.309514 8487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "4c99222c533f49489c067c9a2fb2a670"
format_stamp: "Formatted at 2026-08-08 17:02:05 on dist-test-slave-2j7r"
server_key: "6caa299fc153e72121af1ea0233539f9"
server_key_iv: "911324afbab994c170e5ccc487e82b29"
server_key_version: "encryptionkey@0"
I20260808 17:02:05.309842 8487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "4c99222c533f49489c067c9a2fb2a670"
format_stamp: "Formatted at 2026-08-08 17:02:05 on dist-test-slave-2j7r"
server_key: "6caa299fc153e72121af1ea0233539f9"
server_key_iv: "911324afbab994c170e5ccc487e82b29"
server_key_version: "encryptionkey@0"
I20260808 17:02:05.311003 8487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:05.311642 8509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:05.311796 8487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:05.311877 8487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "4c99222c533f49489c067c9a2fb2a670"
format_stamp: "Formatted at 2026-08-08 17:02:05 on dist-test-slave-2j7r"
server_key: "6caa299fc153e72121af1ea0233539f9"
server_key_iv: "911324afbab994c170e5ccc487e82b29"
server_key_version: "encryptionkey@0"
I20260808 17:02:05.311956 8487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:05.324029 8487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:05.324267 8487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:05.324386 8487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:05.327836 8487 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:39087
I20260808 17:02:05.327867 8598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:39087 every 8 connection(s)
I20260808 17:02:05.328143 8487 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:05.329005 8600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:05.331038 8600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670: Bootstrap starting.
I20260808 17:02:05.331557 8600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:05.331812 8600 log.cc:824] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:05.332302 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 8487
I20260808 17:02:05.332409 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:05.332530 8600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670: No bootstrap required, opened a new log
I20260808 17:02:05.332585 31496 external_mini_cluster.cc:1428] Setting key 468003b5eb79cd0b0b85348a091f13d3
I20260808 17:02:05.334693 8600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99222c533f49489c067c9a2fb2a670" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39087 } }
I20260808 17:02:05.334817 8600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:05.334849 8600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c99222c533f49489c067c9a2fb2a670, State: Initialized, Role: FOLLOWER
I20260808 17:02:05.334960 8600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99222c533f49489c067c9a2fb2a670" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39087 } }
I20260808 17:02:05.335052 8600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:05.335084 8600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:05.335127 8600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:05.335759 8600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99222c533f49489c067c9a2fb2a670" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39087 } }
I20260808 17:02:05.335861 8600 leader_election.cc:302] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c99222c533f49489c067c9a2fb2a670; no voters:
I20260808 17:02:05.336007 8600 leader_election.cc:288] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:05.336046 8608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:05.336197 8608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [term 1 LEADER]: Becoming Leader. State: Replica: 4c99222c533f49489c067c9a2fb2a670, State: Running, Role: LEADER
I20260808 17:02:05.336258 8600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:05.336288 8608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99222c533f49489c067c9a2fb2a670" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39087 } }
I20260808 17:02:05.336592 8608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c99222c533f49489c067c9a2fb2a670. Latest consensus state: current_term: 1 leader_uuid: "4c99222c533f49489c067c9a2fb2a670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99222c533f49489c067c9a2fb2a670" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39087 } } }
I20260808 17:02:05.336674 8608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:05.336598 8613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c99222c533f49489c067c9a2fb2a670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99222c533f49489c067c9a2fb2a670" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 39087 } } }
I20260808 17:02:05.336835 8613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670 [sys.catalog]: This master's current role is: LEADER
W20260808 17:02:05.338053 8630 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:02:05.338119 8630 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:02:05.338179 8618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:05.338281 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:05.333036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38216 (local address 127.30.194.62:39087)
0808 17:02:05.333249 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:05.333257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:05.333282 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:02:05.333694 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:05.333700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:05.333761 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:05.333875 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:05.334079 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.334917 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.336011 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.336610 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.337040 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:05.337433 (+ 393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:05.337439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:05.337439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:05.337635 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:05.337683 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:05.337698 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:05.337770 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:05.337961 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 17:02:05.338036 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260808 17:02:05.338367 8618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:05.339073 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:02:05.339736 8618 catalog_manager.cc:1347] Generated new cluster ID: b662eebc82c4465f9f707184e130d3d8
I20260808 17:02:05.339787 8618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:05.355728 8618 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:05.356145 8618 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:05.358942 8618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c99222c533f49489c067c9a2fb2a670: Generated new TSK 0
I20260808 17:02:05.359072 8618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:02:05.789906 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:05.366969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38220 (local address 127.30.194.62:39087)
0808 17:02:05.367020 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:05.367022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:05.367122 (+ 100us) server_negotiation.cc:407] Connection header received
0808 17:02:05.367252 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:05.367253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:05.367262 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:05.367315 (+ 53us) server_negotiation.cc:226] Negotiated authn=JWT
0808 17:02:05.367444 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.367901 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.368951 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.369150 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.369194 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:05.369383 (+ 189us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 17:02:05.789679 (+420296us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 17:02:05.789844 (+ 165us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38220: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":30}
W20260808 17:02:05.789974 8607 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38220: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 17:02:05.790064 8647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:05.366915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:39087 (local address 127.0.0.1:38220)
0808 17:02:05.367056 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:05.367066 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:05.367153 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:05.367331 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:05.367331 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:05.367333 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 17:02:05.367387 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:05.367388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.367964 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.367965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:05.368882 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:05.368885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:05.369207 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:05.369208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:05.369276 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:05.369281 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 17:02:05.789839 (+420558us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 17:02:05.789963 (+ 124us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.194.62:39087: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260808 17:02:05.790393 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 8487
2026-08-08T17:02:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (603 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (84 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:02:06 dist-test-slave-2j7r krb5kdc[8667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:02:06 dist-test-slave-2j7r krb5kdc[8667](info): set up 2 sockets
Aug 08 17:02:06 dist-test-slave-2j7r krb5kdc[8667](info): commencing operation
krb5kdc: starting...
W20260808 17:02:07.104465 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:02:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:02:07.214300 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:46647
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:46647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:07.293901 8691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:07.294077 8691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:07.294097 8691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:07.295166 8691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:07.295195 8691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:07.295207 8691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:07.295218 8691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:07.295228 8691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:07.296583 8691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:46647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:46647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:07.296765 8691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:07.296928 8691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:07.301298 8701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:07.301369 8691 server_base.cc:1034] running on GCE node
W20260808 17:02:07.301312 8699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:07.301343 8697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:07.301694 8691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:07.301962 8691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:07.303093 8691 hybrid_clock.cc:648] HybridClock initialized: now 1786208527303084 us; error 32 us; skew 500 ppm
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.305418 8691 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:02:07.305814 8691 webserver.cc:460] Webserver started at http://127.30.194.62:43083/ using document root <none> and password file <none>
I20260808 17:02:07.306025 8691 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:07.306067 8691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:07.306162 8691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:07.307168 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "c6eb4e62c92043aaa2173696667f3d1f"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "37929368f4373f42701a714d9efe49a5"
server_key_iv: "c6a883031f3ecabd9ae69b40d8bb0791"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.307459 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "c6eb4e62c92043aaa2173696667f3d1f"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "37929368f4373f42701a714d9efe49a5"
server_key_iv: "c6a883031f3ecabd9ae69b40d8bb0791"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.308696 8691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:02:07.309408 8716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:07.309571 8691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:02:07.309659 8691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "c6eb4e62c92043aaa2173696667f3d1f"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "37929368f4373f42701a714d9efe49a5"
server_key_iv: "c6a883031f3ecabd9ae69b40d8bb0791"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.309724 8691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:07.329442 8691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:07.331514 8691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:07.331631 8691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:07.335587 8691 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:46647
I20260808 17:02:07.335628 8811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:46647 every 8 connection(s)
I20260808 17:02:07.335947 8691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:07.336812 8812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:07.339032 8812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f: Bootstrap starting.
I20260808 17:02:07.339107 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 8691
I20260808 17:02:07.339215 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:07.339403 31496 external_mini_cluster.cc:1428] Setting key 1db8b942de1d15685a305b67b4d4638f
I20260808 17:02:07.339633 8812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:07.339957 8812 log.cc:824] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:07.340597 8812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f: No bootstrap required, opened a new log
I20260808 17:02:07.342723 8812 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6eb4e62c92043aaa2173696667f3d1f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46647 } }
I20260808 17:02:07.342885 8812 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:07.342913 8812 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6eb4e62c92043aaa2173696667f3d1f, State: Initialized, Role: FOLLOWER
I20260808 17:02:07.343030 8812 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6eb4e62c92043aaa2173696667f3d1f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46647 } }
I20260808 17:02:07.343096 8812 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:07.343122 8812 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:07.343155 8812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:07.343667 8812 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6eb4e62c92043aaa2173696667f3d1f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46647 } }
I20260808 17:02:07.343770 8812 leader_election.cc:302] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6eb4e62c92043aaa2173696667f3d1f; no voters:
I20260808 17:02:07.343940 8812 leader_election.cc:288] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:07.344012 8822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:07.344185 8812 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:07.344195 8822 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [term 1 LEADER]: Becoming Leader. State: Replica: c6eb4e62c92043aaa2173696667f3d1f, State: Running, Role: LEADER
I20260808 17:02:07.344391 8822 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6eb4e62c92043aaa2173696667f3d1f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46647 } }
I20260808 17:02:07.345433 8827 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6eb4e62c92043aaa2173696667f3d1f. Latest consensus state: current_term: 1 leader_uuid: "c6eb4e62c92043aaa2173696667f3d1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6eb4e62c92043aaa2173696667f3d1f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46647 } } }
I20260808 17:02:07.345520 8827 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:07.345773 8825 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6eb4e62c92043aaa2173696667f3d1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6eb4e62c92043aaa2173696667f3d1f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46647 } } }
I20260808 17:02:07.345832 8825 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f [sys.catalog]: This master's current role is: LEADER
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
W20260808 17:02:07.346040 8846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:02:07.346094 8846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:02:07.347059 8849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:07.347218 8849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:07.348656 8849 catalog_manager.cc:1347] Generated new cluster ID: d497efc4f855428f9182a5d4772b4f2b
I20260808 17:02:07.348697 8849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:07.350317 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:07.339862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.30.194.62:46647)
0808 17:02:07.340014 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:07.340023 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:07.340055 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:02:07.340453 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:07.340459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:07.340502 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:07.340612 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:07.341381 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.342248 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.343278 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.343662 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.343753 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:07.346525 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:07.346529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:07.346530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:07.346541 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:07.348315 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.348701 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.348704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.348705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.348821 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.349022 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.349023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.349024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.349341 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:07.349487 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:07.349977 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 17:02:07.350110 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260808 17:02:07.361857 8849 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:07.362571 8849 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:07.375694 8849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6eb4e62c92043aaa2173696667f3d1f: Generated new TSK 0
I20260808 17:02:07.375862 8849 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:02:07.405205 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46647
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:07.483879 8855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:07.484078 8855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:07.484100 8855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:07.485188 8855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:07.485226 8855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:07.485250 8855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:07.486658 8855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:07.486842 8855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:07.487007 8855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:07.491556 8871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:07.491580 8868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:07.491566 8867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:07.491784 8855 server_base.cc:1034] running on GCE node
I20260808 17:02:07.491966 8855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:07.492142 8855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:07.493275 8855 hybrid_clock.cc:648] HybridClock initialized: now 1786208527493260 us; error 27 us; skew 500 ppm
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.495662 8855 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:02:07.496071 8855 webserver.cc:460] Webserver started at http://127.30.194.1:44735/ using document root <none> and password file <none>
I20260808 17:02:07.496294 8855 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:07.496344 8855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:07.496474 8855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:07.497493 8855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "afad139fb3cc4dcc802b89adb91f84c3"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "f1eb3bd0eab687264412cb78a9e7c410"
server_key_iv: "026e3d6193c60c4003175c4b0f07d26c"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.497836 8855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "afad139fb3cc4dcc802b89adb91f84c3"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "f1eb3bd0eab687264412cb78a9e7c410"
server_key_iv: "026e3d6193c60c4003175c4b0f07d26c"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.499122 8855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:07.499784 8881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:07.499922 8855 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:07.500006 8855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "afad139fb3cc4dcc802b89adb91f84c3"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "f1eb3bd0eab687264412cb78a9e7c410"
server_key_iv: "026e3d6193c60c4003175c4b0f07d26c"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.500075 8855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:07.513063 8855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:07.514705 8855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:07.514825 8855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:07.515013 8855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:07.515277 8855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:07.515308 8855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:07.515328 8855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:07.515350 8855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:07.520536 8855 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:36915
I20260808 17:02:07.520591 9068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:36915 every 8 connection(s)
I20260808 17:02:07.520849 8855 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:07.528942 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:07.521480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:52509 (local address 127.30.194.62:46647)
0808 17:02:07.521540 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:07.521542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:07.521696 (+ 154us) server_negotiation.cc:407] Connection header received
0808 17:02:07.522547 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:07.522548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:07.522562 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:07.522612 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:07.523174 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.523669 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.524599 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.524818 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.524873 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:07.526700 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:07.526702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:07.526702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:07.526710 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:07.527751 (+ 1041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.528087 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.528088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.528089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.528143 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.528360 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.528361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.528361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.528472 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:07.528528 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:07.528765 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 17:02:07.528853 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:02:07.529067 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:07.521452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:46647 (local address 127.30.194.1:52509)
0808 17:02:07.521589 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:07.521621 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:07.522441 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:07.522672 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:07.522676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:07.522913 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:07.523080 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:07.523084 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.523755 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.523763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:07.524513 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:07.524515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.524887 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.524888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:07.525001 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:07.525367 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:07.525374 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:07.526618 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:07.527818 (+ 1200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:07.527824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:07.527826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:07.528022 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:07.528193 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:07.528194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:07.528195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:07.528301 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:07.528526 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:07.528532 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:07.528652 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 17:02:07.528743 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 17:02:07.528880 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260808 17:02:07.529382 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 8855
I20260808 17:02:07.529490 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:02:07.529613 9070 heartbeater.cc:347] Connected to a master server at 127.30.194.62:46647
I20260808 17:02:07.529690 9070 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:07.529682 31496 external_mini_cluster.cc:1428] Setting key dbc111fac09cad0c6e38e15283cdee3a
I20260808 17:02:07.529820 9070 heartbeater.cc:511] Master 127.30.194.62:46647 requested a full tablet report, sending...
I20260808 17:02:07.530354 8744 ts_manager.cc:194] Registered new tserver with Master: afad139fb3cc4dcc802b89adb91f84c3 (127.30.194.1:36915)
I20260808 17:02:07.531124 8744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:52509
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:02:07.553421 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46647
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:07.631803 9078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:07.631984 9078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:07.632016 9078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:07.633126 9078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:07.633163 9078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:07.633198 9078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:07.634605 9078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:07.634824 9078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:07.635013 9078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:07.639528 9086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:07.639580 9087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:07.639694 9078 server_base.cc:1034] running on GCE node
W20260808 17:02:07.639580 9089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:07.639928 9078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:07.640147 9078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:07.641290 9078 hybrid_clock.cc:648] HybridClock initialized: now 1786208527641274 us; error 30 us; skew 500 ppm
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.643579 9078 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:02:07.643986 9078 webserver.cc:460] Webserver started at http://127.30.194.2:46329/ using document root <none> and password file <none>
I20260808 17:02:07.644194 9078 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:07.644248 9078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:07.644388 9078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:07.645416 9078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "6dc4c3ec3f2448caaa38eac064296d9b"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "58878975c9de83fab6609dd131f27511"
server_key_iv: "4d9b57dc6844f2f0093486192591eaf7"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.645736 9078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "6dc4c3ec3f2448caaa38eac064296d9b"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "58878975c9de83fab6609dd131f27511"
server_key_iv: "4d9b57dc6844f2f0093486192591eaf7"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.646935 9078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:07.647579 9098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:07.647725 9078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:07.647802 9078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "6dc4c3ec3f2448caaa38eac064296d9b"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "58878975c9de83fab6609dd131f27511"
server_key_iv: "4d9b57dc6844f2f0093486192591eaf7"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.647864 9078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:07.655989 9078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:07.657778 9078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:07.657898 9078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:07.658073 9078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:07.658341 9078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:07.658372 9078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:07.658397 9078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:07.658413 9078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:07.663771 9078 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:35577
I20260808 17:02:07.663815 9292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:35577 every 8 connection(s)
I20260808 17:02:07.664060 9078 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:02:07.667411 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 9078
I20260808 17:02:07.667518 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:02:07.667703 31496 external_mini_cluster.cc:1428] Setting key 72ada35fe3f4a9d09c4ab7fb1bd85f3b
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:07.672912 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:07.664675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:47749 (local address 127.30.194.62:46647)
0808 17:02:07.664727 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:07.664729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:07.664890 (+ 161us) server_negotiation.cc:407] Connection header received
0808 17:02:07.665810 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:07.665812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:07.665824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:07.665870 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:07.666443 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.666938 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.668179 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.668377 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.668435 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:07.670531 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:07.670533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:07.670533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:07.670541 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:07.671697 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.672044 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.672046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.672046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.672097 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.672343 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.672344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.672344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.672446 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:07.672502 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:07.672739 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 17:02:07.672823 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:02:07.673038 9296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:07.664627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:46647 (local address 127.30.194.2:47749)
0808 17:02:07.664787 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:07.664820 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:07.665704 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:07.665955 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:07.665959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:07.666181 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:07.666357 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:07.666361 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.667018 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.667019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:07.668103 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:07.668106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.668460 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.668462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:07.668607 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:07.669009 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:07.669016 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:07.670447 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:07.671769 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:07.671775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:07.671777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:07.671980 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:07.672151 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:07.672151 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:07.672152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:07.672275 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:07.672500 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:07.672506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:07.672627 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 17:02:07.672709 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 17:02:07.672847 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260808 17:02:07.673413 9293 heartbeater.cc:347] Connected to a master server at 127.30.194.62:46647
I20260808 17:02:07.673498 9293 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:07.673650 9293 heartbeater.cc:511] Master 127.30.194.62:46647 requested a full tablet report, sending...
I20260808 17:02:07.673976 8744 ts_manager.cc:194] Registered new tserver with Master: 6dc4c3ec3f2448caaa38eac064296d9b (127.30.194.2:35577)
I20260808 17:02:07.674321 8744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:47749
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:02:07.691744 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46647
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:07.771484 9300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:07.771667 9300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:07.771699 9300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:07.772814 9300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:07.772858 9300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:07.772886 9300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:07.774348 9300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:07.774551 9300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:07.774778 9300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:07.779383 9314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:07.779392 9317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:07.779467 9300 server_base.cc:1034] running on GCE node
W20260808 17:02:07.779426 9311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:07.779789 9300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:07.780014 9300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:07.781139 9300 hybrid_clock.cc:648] HybridClock initialized: now 1786208527781124 us; error 27 us; skew 500 ppm
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.783819 9300 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:02:07.784199 9300 webserver.cc:460] Webserver started at http://127.30.194.3:34731/ using document root <none> and password file <none>
I20260808 17:02:07.784408 9300 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:07.784456 9300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:07.784575 9300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:07.785565 9300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "84cc7aa62f8744828b0bee1897be2d00"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "7344a127fd7e55b3fc5b4555331f1e7a"
server_key_iv: "f1a5256c99f9ec84fb819c66b712d594"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.785874 9300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "84cc7aa62f8744828b0bee1897be2d00"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "7344a127fd7e55b3fc5b4555331f1e7a"
server_key_iv: "f1a5256c99f9ec84fb819c66b712d594"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.787070 9300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:07.787694 9331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:07.787868 9300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:07.787946 9300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "84cc7aa62f8744828b0bee1897be2d00"
format_stamp: "Formatted at 2026-08-08 17:02:07 on dist-test-slave-2j7r"
server_key: "7344a127fd7e55b3fc5b4555331f1e7a"
server_key_iv: "f1a5256c99f9ec84fb819c66b712d594"
server_key_version: "encryptionkey@0"
I20260808 17:02:07.788002 9300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:07.807183 9300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:07.808862 9300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:07.808960 9300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:07.809149 9300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:07.809412 9300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:07.809455 9300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:07.809476 9300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:07.809489 9300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:07.814841 9300 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:35043
I20260808 17:02:07.814913 9511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:35043 every 8 connection(s)
I20260808 17:02:07.815132 9300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:02:07.815812 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 9300
I20260808 17:02:07.815907 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:02:07.816071 31496 external_mini_cluster.cc:1428] Setting key 596e8b0dd7547f99d6716f7f19353450
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:07.823649 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:07.815868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:37355 (local address 127.30.194.62:46647)
0808 17:02:07.815924 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:07.815926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:07.816144 (+ 218us) server_negotiation.cc:407] Connection header received
0808 17:02:07.817272 (+ 1128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:07.817273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:07.817283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:07.817326 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:07.817877 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.818363 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.819285 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.819496 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.819549 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:07.821510 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:07.821512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:07.821512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:07.821518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:07.822606 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.822955 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.822955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.822956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.822999 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.823174 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.823175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.823175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.823286 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:07.823332 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:07.823532 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 17:02:07.823586 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 17:02:07.823819 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:07.815810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:46647 (local address 127.30.194.3:37355)
0808 17:02:07.816043 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:07.816072 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:07.817166 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:07.817386 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:07.817389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:07.817634 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:07.817817 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:07.817821 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.818423 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.818423 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:07.819196 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:07.819198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.819565 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.819566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:07.819657 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:07.820100 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:07.820109 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:07.821410 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:07.822672 (+ 1262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:07.822678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:07.822680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:07.822893 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:07.823043 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:07.823043 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:07.823044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:07.823129 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:07.823333 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:07.823336 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:07.823436 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 17:02:07.823517 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:02:07.823628 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":87,"threads_started":1}
I20260808 17:02:07.824258 9512 heartbeater.cc:347] Connected to a master server at 127.30.194.62:46647
I20260808 17:02:07.824343 9512 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:07.824496 9512 heartbeater.cc:511] Master 127.30.194.62:46647 requested a full tablet report, sending...
I20260808 17:02:07.824828 8744 ts_manager.cc:194] Registered new tserver with Master: 84cc7aa62f8744828b0bee1897be2d00 (127.30.194.3:35043)
I20260808 17:02:07.825255 8744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:37355
I20260808 17:02:07.827700 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/test-admin.keytab.
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.842641 31496 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 17:02:07.844043 9522 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:07.843373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.30.194.1:36915)
0808 17:02:07.843552 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:07.843556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:07.843565 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:07.843587 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:07.843588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:07.843601 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:07.843664 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:07.843978 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35748: BlockingRecv error: recv got EOF from 127.0.0.1:35748 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
W20260808 17:02:07.844220 31496 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5cc' of type FILE
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.845184 31496 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.846047 31496 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:02:07.853011 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:07.846534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.30.194.1:36915)
0808 17:02:07.846576 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:07.846578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:07.846589 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:02:07.846641 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:07.846642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:07.846652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:07.846708 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:07.847028 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.847741 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.848408 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.848723 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.848784 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:07.850746 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:07.850753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:07.850754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:07.850760 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:07.852086 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.852403 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.852406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.852407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.852466 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.852605 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.852605 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.852606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.852717 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:07.852806 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:07.852855 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 17:02:07.852915 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 17:02:07.854554 9522 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:07.854047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35768 (local address 127.30.194.1:36915)
0808 17:02:07.854081 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:07.854082 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:07.854109 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:02:07.854151 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:07.854151 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:07.854159 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:07.854196 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:07.854512 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35768: BlockingRecv error: recv got EOF from 127.0.0.1:35768 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:02:07.854635 31496 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5cc' of type FILE
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.855443 31496 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.856175 31496 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:02:07.861868 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:07.856584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35780 (local address 127.30.194.1:36915)
0808 17:02:07.856619 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:07.856621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:07.856670 (+ 49us) server_negotiation.cc:407] Connection header received
0808 17:02:07.856703 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:07.856704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:07.856711 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:07.856754 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:07.857042 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.857531 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.858125 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.858323 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.858368 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:07.860007 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:07.860008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:07.860009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:07.860014 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:07.861014 (+ 1000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.861383 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.861384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.861384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.861431 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.861557 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.861557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.861558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.861626 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:07.861680 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:07.861745 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 17:02:07.861798 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 17:02:07.863255 9522 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:07.862761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35790 (local address 127.30.194.1:36915)
0808 17:02:07.862796 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:07.862797 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:07.862806 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:07.862872 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:07.862872 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:07.862881 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:07.862939 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:07.863215 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35790: BlockingRecv error: recv got EOF from 127.0.0.1:35790 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 17:02:07.863323 31496 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208501350379-31496-0/krb5kdc/krb5cc' of type FILE
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.864161 31496 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:07.864913 31496 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[8667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:02:07.870620 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:07.865391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35806 (local address 127.30.194.1:36915)
0808 17:02:07.865423 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:07.865425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:07.865454 (+ 29us) server_negotiation.cc:407] Connection header received
0808 17:02:07.865493 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:07.865494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:07.865501 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:07.865546 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:07.865863 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.866303 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.866874 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:07.867062 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:07.867105 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:07.868692 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:07.868693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:07.868693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:07.868698 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:07.869860 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.870129 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.870130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.870130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.870176 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:07.870328 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:07.870328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:07.870328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:07.870400 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:07.870456 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:07.870523 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 17:02:07.870571 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 17:02:07.871100 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 8855
I20260808 17:02:07.885741 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 9078
I20260808 17:02:07.912760 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 9300
I20260808 17:02:07.922854 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 8691
2026-08-08T17:02:07Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1957 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[9546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[9546](info): set up 2 sockets
Aug 08 17:02:07 dist-test-slave-2j7r krb5kdc[9546](info): commencing operation
krb5kdc: starting...
W20260808 17:02:09.065536 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:02:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.62.keytab.
Entry for principal oryx/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.62.keytab.
I20260808 17:02:09.175552 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:35661
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.62.keytab
--principal=oryx/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:35661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:09.256906 9568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:09.257119 9568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:09.257141 9568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:09.258271 9568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:09.258312 9568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:09.258324 9568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:09.258347 9568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:09.258370 9568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:09.259835 9568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:35661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.62.keytab
--principal=oryx/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:35661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:09.260063 9568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:09.260278 9568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:09.264816 9583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:09.264794 9577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:09.264813 9576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:09.265015 9568 server_base.cc:1034] running on GCE node
I20260808 17:02:09.265281 9568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:09.265568 9568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:09.266709 9568 hybrid_clock.cc:648] HybridClock initialized: now 1786208529266688 us; error 38 us; skew 500 ppm
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:09.269227 9568 init.cc:377] Logged in from keytab as oryx/127.30.194.62@KRBTEST.COM (short username oryx)
I20260808 17:02:09.269651 9568 webserver.cc:460] Webserver started at http://127.30.194.62:36227/ using document root <none> and password file <none>
I20260808 17:02:09.269873 9568 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:09.269917 9568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:09.270035 9568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:09.271030 9568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "b46b984f705b49c19aec28869ecc7594"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "620516f02de44be460569e39ece891ef"
server_key_iv: "bcde8910ed3b976a5863ec265481dc02"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.271354 9568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "b46b984f705b49c19aec28869ecc7594"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "620516f02de44be460569e39ece891ef"
server_key_iv: "bcde8910ed3b976a5863ec265481dc02"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.272588 9568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:02:09.273243 9592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.273404 9568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:09.273494 9568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "b46b984f705b49c19aec28869ecc7594"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "620516f02de44be460569e39ece891ef"
server_key_iv: "bcde8910ed3b976a5863ec265481dc02"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.273561 9568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:09.284125 9568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:09.285794 9568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:09.285921 9568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:09.289719 9568 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:35661
I20260808 17:02:09.289762 9681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:35661 every 8 connection(s)
I20260808 17:02:09.290079 9568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:09.291035 9682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:09.293246 9682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594: Bootstrap starting.
I20260808 17:02:09.293798 9682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:09.294051 9682 log.cc:824] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:09.294673 9682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594: No bootstrap required, opened a new log
I20260808 17:02:09.296206 9682 raft_consensus.cc:348] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } }
I20260808 17:02:09.296350 9682 raft_consensus.cc:374] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:09.296371 9682 raft_consensus.cc:729] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46b984f705b49c19aec28869ecc7594, State: Initialized, Role: FOLLOWER
I20260808 17:02:09.296494 9682 consensus_queue.cc:260] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } }
I20260808 17:02:09.296562 9682 raft_consensus.cc:388] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:09.296597 9682 raft_consensus.cc:482] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:09.296644 9682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:09.297452 9682 raft_consensus.cc:504] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } }
I20260808 17:02:09.297539 9682 leader_election.cc:302] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b46b984f705b49c19aec28869ecc7594; no voters:
I20260808 17:02:09.297674 9682 leader_election.cc:288] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:09.297737 9686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:09.297896 9686 raft_consensus.cc:686] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 1 LEADER]: Becoming Leader. State: Replica: b46b984f705b49c19aec28869ecc7594, State: Running, Role: LEADER
I20260808 17:02:09.297901 9682 sys_catalog.cc:564] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:09.298012 9686 consensus_queue.cc:237] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } }
I20260808 17:02:09.298454 9687 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } }
I20260808 17:02:09.298475 9688 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b46b984f705b49c19aec28869ecc7594. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } }
I20260808 17:02:09.298557 9687 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:09.298563 9688 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:09.298771 9703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:09.299305 9703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:09.299772 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 9568
I20260808 17:02:09.299863 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:09.300071 31496 external_mini_cluster.cc:1428] Setting key 482f3cda07ce61ce4a7cb413c6c2bbc5
I20260808 17:02:09.300984 9703 catalog_manager.cc:1347] Generated new cluster ID: d4aac8015e2d4470b35e4009d8309005
I20260808 17:02:09.301035 9703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:09.311874 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.300548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.30.194.62:35661)
0808 17:02:09.300701 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:09.300710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:09.300747 (+ 37us) server_negotiation.cc:407] Connection header received
0808 17:02:09.300920 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:09.300927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:09.300984 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:09.301848 (+ 864us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:09.301935 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.303264 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.304286 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.304879 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.304973 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.307753 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:09.307757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:09.307757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:09.307770 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:09.309756 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.310112 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.310115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.310115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.310228 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.310478 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.310479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.310479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.310804 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:09.310956 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:09.311479 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 17:02:09.311619 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
I20260808 17:02:09.315434 9703 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:09.316102 9703 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:09.321527 9703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594: Generated new TSK 0
I20260808 17:02:09.321703 9703 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab.
Entry for principal oryx/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab.
I20260808 17:02:09.353590 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35661
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab
--principal=oryx/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:09.433467 9725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:09.433652 9725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:09.433673 9725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:09.435065 9725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:09.435106 9725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:09.435130 9725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:09.436517 9725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab
--principal=oryx/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:09.436729 9725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:09.436949 9725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:09.441705 9731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:09.441720 9732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:09.441856 9734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:09.441916 9725 server_base.cc:1034] running on GCE node
I20260808 17:02:09.442091 9725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:09.442298 9725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:09.443444 9725 hybrid_clock.cc:648] HybridClock initialized: now 1786208529443438 us; error 36 us; skew 500 ppm
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:09.445801 9725 init.cc:377] Logged in from keytab as oryx/127.30.194.1@KRBTEST.COM (short username oryx)
I20260808 17:02:09.446206 9725 webserver.cc:460] Webserver started at http://127.30.194.1:44543/ using document root <none> and password file <none>
I20260808 17:02:09.446375 9725 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:09.446412 9725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:09.446501 9725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:09.447389 9725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "b4b7243cd2ab4fd39a4a4968c110d86e"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "89ca2f1babd5ef292891a1515b3d5ed5"
server_key_iv: "e4eaf3a91ae34bca5cd131287538ea0a"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.447679 9725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "b4b7243cd2ab4fd39a4a4968c110d86e"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "89ca2f1babd5ef292891a1515b3d5ed5"
server_key_iv: "e4eaf3a91ae34bca5cd131287538ea0a"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.448905 9725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:02:09.449599 9745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.449738 9725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.449819 9725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "b4b7243cd2ab4fd39a4a4968c110d86e"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "89ca2f1babd5ef292891a1515b3d5ed5"
server_key_iv: "e4eaf3a91ae34bca5cd131287538ea0a"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.449878 9725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:09.466758 9725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:09.468699 9725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:09.468825 9725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:09.469019 9725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:09.469306 9725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:09.469340 9725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.469360 9725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:09.469372 9725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.474751 9725 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:37889
I20260808 17:02:09.474803 9911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:37889 every 8 connection(s)
I20260808 17:02:09.475066 9725 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:02:09.478165 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 9725
I20260808 17:02:09.478266 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:02:09.478472 31496 external_mini_cluster.cc:1428] Setting key a3e0053181ffc50302bb8b7b711774ff
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:09.483989 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.475680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:57149 (local address 127.30.194.62:35661)
0808 17:02:09.475771 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:09.475774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:09.475895 (+ 121us) server_negotiation.cc:407] Connection header received
0808 17:02:09.476346 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:09.476347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:09.476361 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:09.476410 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:09.477322 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.477812 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.478993 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.479250 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.479309 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.481410 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:09.481411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:09.481412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:09.481420 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:09.482746 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.483089 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.483091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.483091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.483152 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.483360 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.483361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.483362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.483494 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:09.483565 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:09.483810 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 17:02:09.483894 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:02:09.484028 9915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.475647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.1:57149)
0808 17:02:09.475788 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:09.475822 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:09.476265 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:09.476459 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:09.476462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:09.477015 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:09.477247 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.477252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.477873 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.477874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.478916 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.478919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.479334 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.479335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.479464 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.479880 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:09.479889 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:09.481328 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:09.482818 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.482825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.482826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.483022 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.483210 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.483211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.483211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.483302 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.483575 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:09.483580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:09.483690 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 17:02:09.483772 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 17:02:09.483881 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260808 17:02:09.484447 9912 heartbeater.cc:347] Connected to a master server at 127.30.194.62:35661
I20260808 17:02:09.484526 9912 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:09.484685 9912 heartbeater.cc:511] Master 127.30.194.62:35661 requested a full tablet report, sending...
I20260808 17:02:09.485201 9623 ts_manager.cc:194] Registered new tserver with Master: b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889)
I20260808 17:02:09.485746 9623 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:57149
WARNING: no policy specified for oryx/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab.
Entry for principal oryx/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab.
I20260808 17:02:09.501552 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35661
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab
--principal=oryx/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:09.580300 9918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:09.580485 9918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:09.580506 9918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:09.581610 9918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:09.581650 9918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:09.581673 9918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:09.583073 9918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab
--principal=oryx/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:09.583266 9918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:09.583465 9918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:09.588074 9931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:09.588071 9928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:09.588104 9929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:09.588346 9918 server_base.cc:1034] running on GCE node
I20260808 17:02:09.588538 9918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:09.588749 9918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:09.589879 9918 hybrid_clock.cc:648] HybridClock initialized: now 1786208529589867 us; error 29 us; skew 500 ppm
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:09.592167 9918 init.cc:377] Logged in from keytab as oryx/127.30.194.2@KRBTEST.COM (short username oryx)
I20260808 17:02:09.592569 9918 webserver.cc:460] Webserver started at http://127.30.194.2:45083/ using document root <none> and password file <none>
I20260808 17:02:09.592760 9918 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:09.592805 9918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:09.592921 9918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:09.593889 9918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "925364577a1542148cb6c46abf5ab9ec"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "9681f678dbb53fe2b100d1490e2c4748"
server_key_iv: "d4310a9c26e3b3d77873610b06da750b"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.594200 9918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "925364577a1542148cb6c46abf5ab9ec"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "9681f678dbb53fe2b100d1490e2c4748"
server_key_iv: "d4310a9c26e3b3d77873610b06da750b"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.595427 9918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:09.596088 9943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.596240 9918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:09.596318 9918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "925364577a1542148cb6c46abf5ab9ec"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "9681f678dbb53fe2b100d1490e2c4748"
server_key_iv: "d4310a9c26e3b3d77873610b06da750b"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.596381 9918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:09.617007 9918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:09.618588 9918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:09.618721 9918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:09.618908 9918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:09.619175 9918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:09.619207 9918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.619233 9918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:09.619249 9918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.624694 9918 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:35543
I20260808 17:02:09.624754 10125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:35543 every 8 connection(s)
I20260808 17:02:09.625020 9918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:02:09.625765 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 9918
I20260808 17:02:09.625870 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:02:09.626052 31496 external_mini_cluster.cc:1428] Setting key bcabdc52f19f15c89b2afb6324066d62
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:09.633338 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.626242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:43497 (local address 127.30.194.62:35661)
0808 17:02:09.626309 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:09.626311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:09.626321 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:09.626384 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:09.626385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:09.626396 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:09.626441 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:09.627295 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.627779 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.628681 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.628882 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.628936 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.630874 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:09.630875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:09.630876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:09.630883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:09.632095 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.632445 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.632446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.632447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.632501 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.632707 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.632708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.632708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.632816 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:09.632871 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:09.633155 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 17:02:09.633247 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:02:09.633442 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.625610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.2:43497)
0808 17:02:09.625774 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:09.625805 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:09.626287 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:09.626513 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:09.626517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:09.627038 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:09.627203 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.627207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.627857 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.627859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.628599 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.628601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.628949 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.628950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.629066 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.629519 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:09.629527 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:09.630790 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:09.632165 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.632171 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.632173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.632380 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.632555 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.632556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.632556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.632653 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.632869 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:09.632875 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:09.632997 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 17:02:09.633079 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 17:02:09.633245 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260808 17:02:09.633823 10127 heartbeater.cc:347] Connected to a master server at 127.30.194.62:35661
I20260808 17:02:09.633913 10127 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:09.634068 10127 heartbeater.cc:511] Master 127.30.194.62:35661 requested a full tablet report, sending...
I20260808 17:02:09.634392 9623 ts_manager.cc:194] Registered new tserver with Master: 925364577a1542148cb6c46abf5ab9ec (127.30.194.2:35543)
WARNING: no policy specified for oryx/127.30.194.3@KRBTEST.COM; defaulting to no policy
I20260808 17:02:09.634773 9623 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:43497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab.
Entry for principal oryx/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab.
I20260808 17:02:09.649121 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35661
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab
--principal=oryx/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:09.728179 10135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:09.728380 10135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:09.728413 10135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:09.729552 10135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:09.729597 10135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:09.729635 10135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:09.731344 10135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab
--principal=oryx/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:09.731642 10135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:09.731892 10135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:09.737705 10146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:09.737727 10142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:09.737824 10141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:09.738868 10135 server_base.cc:1034] running on GCE node
I20260808 17:02:09.739401 10135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:09.739665 10135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:09.740840 10135 hybrid_clock.cc:648] HybridClock initialized: now 1786208529740814 us; error 32 us; skew 500 ppm
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:09.743894 10135 init.cc:377] Logged in from keytab as oryx/127.30.194.3@KRBTEST.COM (short username oryx)
I20260808 17:02:09.744433 10135 webserver.cc:460] Webserver started at http://127.30.194.3:43371/ using document root <none> and password file <none>
I20260808 17:02:09.744691 10135 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:09.744764 10135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:09.744910 10135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:09.746328 10135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "99d1b745ca0b424b920ed368038dc8d2"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "987ff17db7fe3e915273051ee3d1e691"
server_key_iv: "ecc29a3e35b41900e94046b67d54d193"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.746752 10135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "99d1b745ca0b424b920ed368038dc8d2"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "987ff17db7fe3e915273051ee3d1e691"
server_key_iv: "ecc29a3e35b41900e94046b67d54d193"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.748646 10135 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:09.749543 10156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.749687 10135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.749774 10135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "99d1b745ca0b424b920ed368038dc8d2"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "987ff17db7fe3e915273051ee3d1e691"
server_key_iv: "ecc29a3e35b41900e94046b67d54d193"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.749840 10135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:09.764091 10135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:09.766278 10135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:09.766412 10135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:09.766623 10135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:09.766928 10135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:09.766969 10135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.766997 10135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:09.767019 10135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.773573 10135 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:33505
I20260808 17:02:09.773638 10337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:33505 every 8 connection(s)
I20260808 17:02:09.773892 10135 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:09.783500 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 10135
I20260808 17:02:09.783583 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.774572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:54983 (local address 127.30.194.62:35661)
0808 17:02:09.774636 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:09.774638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:09.774789 (+ 151us) server_negotiation.cc:407] Connection header received
0808 17:02:09.775289 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:09.775291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:09.775305 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:09.775369 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:09.776419 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.777210 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.778598 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.778891 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.778971 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.780873 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:09.780875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:09.780875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:09.780884 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:09.782460 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.782815 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.782816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.782817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.782869 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.783046 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.783047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.783047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.783143 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:09.783194 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:09.783447 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 17:02:09.783508 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:02:09.783607 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:02:09.783792 31496 external_mini_cluster.cc:1428] Setting key b255db579dd414bb78592f34c9fbccbb
I20260808 17:02:09.783790 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.774503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.3:54983)
0808 17:02:09.774663 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:09.774706 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:09.775179 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:09.775456 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:09.775461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:09.776118 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:09.776341 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.776346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.777301 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.777302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.778505 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.778507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.778971 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.778972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.779102 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.779509 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:09.779517 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:09.780786 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:09.782537 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.782544 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.782545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.782751 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.782914 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.782915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.782915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.783002 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.783186 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:09.783188 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:09.783318 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 17:02:09.783430 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 17:02:09.783584 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260808 17:02:09.784417 10339 heartbeater.cc:347] Connected to a master server at 127.30.194.62:35661
I20260808 17:02:09.784509 10339 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:09.784684 10339 heartbeater.cc:511] Master 127.30.194.62:35661 requested a full tablet report, sending...
I20260808 17:02:09.785109 9622 ts_manager.cc:194] Registered new tserver with Master: 99d1b745ca0b424b920ed368038dc8d2 (127.30.194.3:33505)
I20260808 17:02:09.785477 9622 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:54983
I20260808 17:02:09.795384 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.61.keytab.
Entry for principal oryx/127.30.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.61.keytab.
I20260808 17:02:09.819289 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.61:42613
--webserver_interface=127.30.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.61.keytab
--principal=oryx/127.30.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:35661,127.30.194.61:42613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:09.898324 10348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:09.898535 10348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:09.898563 10348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:09.899768 10348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:09.899809 10348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:09.899828 10348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:09.899843 10348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:09.899858 10348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:09.901329 10348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:35661,127.30.194.61:42613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.61.keytab
--principal=oryx/127.30.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.61:42613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
W20260808 17:02:09.901435 10348 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.194.62:35661,127.30.194.61:42613'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 17:02:09.901578 10348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:09.901762 10348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:09.906179 10358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:09.906189 10359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:09.906191 10361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:09.906369 10348 server_base.cc:1034] running on GCE node
I20260808 17:02:09.906627 10348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:09.906899 10348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:09.908034 10348 hybrid_clock.cc:648] HybridClock initialized: now 1786208529908022 us; error 29 us; skew 500 ppm
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:09.910379 10348 init.cc:377] Logged in from keytab as oryx/127.30.194.61@KRBTEST.COM (short username oryx)
I20260808 17:02:09.910785 10348 webserver.cc:460] Webserver started at http://127.30.194.61:38561/ using document root <none> and password file <none>
I20260808 17:02:09.910969 10348 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:09.911015 10348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:09.911154 10348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:09.912113 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data/instance:
uuid: "c163d97601c94054ab26aacaf129394a"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "63ae114d17e4b2dae54344d407dcd358"
server_key_iv: "efd350670040f14451363dc490cf3427"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.912458 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal/instance:
uuid: "c163d97601c94054ab26aacaf129394a"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "63ae114d17e4b2dae54344d407dcd358"
server_key_iv: "efd350670040f14451363dc490cf3427"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.913753 10348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:02:09.914387 10373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:09.914573 10348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:09.914670 10348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal
uuid: "c163d97601c94054ab26aacaf129394a"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "63ae114d17e4b2dae54344d407dcd358"
server_key_iv: "efd350670040f14451363dc490cf3427"
server_key_version: "encryptionkey@0"
I20260808 17:02:09.914738 10348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:09.933315 10348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:09.935307 10348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:09.935415 10348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:09.939043 10348 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.61:42613
I20260808 17:02:09.939075 10455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.61:42613 every 8 connection(s)
I20260808 17:02:09.939374 10348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
I20260808 17:02:09.940289 10456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:09.941761 10456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:02:09.943421 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 10348
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.61@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.194.61@KRBTEST.COM
I20260808 17:02:09.951103 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.942367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55610 (local address 127.30.194.62:35661)
0808 17:02:09.942426 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:09.942428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:09.942598 (+ 170us) server_negotiation.cc:407] Connection header received
0808 17:02:09.943062 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:09.943064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:09.943077 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:09.943141 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:09.943865 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.944483 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.945657 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.945984 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.946043 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.948542 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:09.948543 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:09.948543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:09.948551 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:09.949871 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.950223 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.950224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.950224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.950276 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.950489 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.950490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.950490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.950629 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:09.950686 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:09.950931 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 17:02:09.951008 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:02:09.951310 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.942333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:55610)
0808 17:02:09.942488 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:09.942526 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:09.942942 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:09.943194 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:09.943197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:09.943504 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:09.943756 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.943761 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.944587 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.944588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.945572 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.945574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.946062 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.946063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.946189 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.946640 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:09.946650 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:09.948384 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:09.949946 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.949952 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.949954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.950155 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.950329 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.950330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.950330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.950434 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.950688 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:09.950694 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:09.950817 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 17:02:09.950903 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 17:02:09.951094 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260808 17:02:09.951834 10456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:02:09.952581 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.943973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59922 (local address 127.30.194.61:42613)
0808 17:02:09.944137 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:09.944142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:09.944211 (+ 69us) server_negotiation.cc:407] Connection header received
0808 17:02:09.944254 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:09.944256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:09.944277 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:09.944349 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:09.945048 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.945928 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.946866 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.947330 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.947401 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.949783 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:09.949791 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:09.949791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:09.949798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:09.951184 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.951559 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.951562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.951563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.951627 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.951818 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.951819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.951819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.952028 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:09.952170 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:09.952389 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 17:02:09.952491 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.61@KRBTEST.COM for oryx/127.30.194.61@KRBTEST.COM
I20260808 17:02:09.958415 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.952039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42613 (local address 127.0.0.1:59934)
0808 17:02:09.952208 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:09.952217 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:09.952272 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:09.952401 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:09.952401 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:09.952557 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:09.952604 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.952605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.953456 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.953457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.953936 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.953938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.954269 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.954270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.954329 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.954655 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:09.954660 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:09.955661 (+ 1001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:09.957569 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.957570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.957571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.957715 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.957880 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.957881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.957881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.957932 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.958126 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:09.958127 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:09.958162 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 17:02:09.958222 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:02:09.958320 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260808 17:02:09.958499 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.952098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59934 (local address 127.30.194.61:42613)
0808 17:02:09.952216 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:09.952217 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:09.952259 (+ 42us) server_negotiation.cc:407] Connection header received
0808 17:02:09.952322 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:09.952323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:09.952331 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:09.952396 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:09.952673 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.953386 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.954002 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.954209 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.954259 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.955733 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:09.955734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:09.955734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:09.955748 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:09.957496 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.957775 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.957776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.957776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.957828 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.957981 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.957981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.957982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.958075 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:09.958127 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:09.958247 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 17:02:09.958370 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":100,"thread_start_us":80,"threads_started":1}
I20260808 17:02:09.959549 10456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Bootstrap starting.
I20260808 17:02:09.960103 10456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:09.960340 10456 log.cc:824] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:09.960959 10456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: No bootstrap required, opened a new log
I20260808 17:02:09.962275 10456 raft_consensus.cc:348] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } }
I20260808 17:02:09.962378 10456 raft_consensus.cc:374] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:09.962409 10456 raft_consensus.cc:729] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c163d97601c94054ab26aacaf129394a, State: Initialized, Role: FOLLOWER
I20260808 17:02:09.962523 10456 consensus_queue.cc:260] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } }
I20260808 17:02:09.962790 10456 sys_catalog.cc:564] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:09.962749 10467 sys_catalog.cc:455] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } } }
I20260808 17:02:09.962850 10467 sys_catalog.cc:458] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: This master's current role is: FOLLOWER
W20260808 17:02:09.964272 10486 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:02:09.964323 10486 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:02:09.965745 10348 master_runner.cc:428] Detected that this master c163d97601c94054ab26aacaf129394a is joining an existing cluster
I20260808 17:02:09.965819 10348 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.194.61:42613
I20260808 17:02:09.972816 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.966688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:55618)
0808 17:02:09.966886 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:09.966896 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:09.966952 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:09.967111 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:09.967112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:09.967295 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:09.967375 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.967377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.968270 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.968271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.968830 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.968831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.969194 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.969195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.969250 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.969611 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:09.969615 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:09.970008 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:09.971689 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.971690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.971690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.971935 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.972153 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.972154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.972154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.972221 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.972492 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:09.972492 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:09.972533 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 17:02:09.972614 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:02:09.972707 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":27,"thread_start_us":84,"threads_started":1}
I20260808 17:02:09.972831 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.966793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.30.194.62:35661)
0808 17:02:09.966828 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:09.966830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:09.966955 (+ 125us) server_negotiation.cc:407] Connection header received
0808 17:02:09.967003 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:09.967004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:09.967013 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:09.967058 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:09.967475 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.968191 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.968907 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.969121 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.969177 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.970111 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:09.970113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:09.970113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:09.970119 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:09.971583 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.972014 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.972015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.972015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.972079 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.972287 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.972287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.972288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.972421 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:09.972495 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:09.972656 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 17:02:09.972740 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:02:09.973501 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.966693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42613 (local address 127.0.0.1:59936)
0808 17:02:09.967005 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:09.967011 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:09.967046 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:09.967142 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:09.967143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:09.967323 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:09.967370 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.967371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.968235 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.968236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.969140 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.969142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.969596 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.969597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.969648 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.969968 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:09.969973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:09.970341 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:09.972324 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.972325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.972325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.972572 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.972763 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.972764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.972764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.972820 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.973156 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:09.973156 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:09.973210 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 17:02:09.973292 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 17:02:09.973385 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"spinlock_wait_cycles":8448,"thread_start_us":102,"threads_started":1}
I20260808 17:02:09.973814 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.966856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59936 (local address 127.30.194.61:42613)
0808 17:02:09.966890 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:09.966892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:09.967054 (+ 162us) server_negotiation.cc:407] Connection header received
0808 17:02:09.967088 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:09.967089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:09.967097 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:09.967140 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:09.967433 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.968149 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.969233 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.969496 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.969556 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.970763 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:09.970765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:09.970765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:09.970772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:09.972243 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.972640 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.972640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.972641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.972696 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.972881 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.972882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.972882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.973024 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:09.973132 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:09.973664 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 17:02:09.973740 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 17:02:09 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.62@KRBTEST.COM for oryx/127.30.194.61@KRBTEST.COM
I20260808 17:02:09.981462 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.975336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59940 (local address 127.30.194.61:42613)
0808 17:02:09.975381 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:09.975383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:09.975523 (+ 140us) server_negotiation.cc:407] Connection header received
0808 17:02:09.975592 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:09.975593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:09.975602 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:09.975646 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:09.976024 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.976514 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.977366 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.977550 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.977601 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.979297 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:09.979298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:09.979299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:09.979305 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:09.980414 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.980745 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.980746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.980746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.980796 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:09.980955 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:09.980956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:09.980956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:09.981052 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:09.981129 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:09.981310 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 17:02:09.981380 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:02:09.981477 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:09.975263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42613 (local address 127.0.0.1:59940)
0808 17:02:09.975433 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:09.975462 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:09.975530 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:09.975650 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:09.975653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:09.975909 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:09.975951 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.975953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.976578 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.976579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.977254 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:09.977257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:09.977609 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:09.977610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:09.977687 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:09.978054 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:09.978058 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:09.979220 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:09.980485 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.980491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.980493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.980673 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.980850 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:09.980850 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:09.980851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:09.980906 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:09.981123 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:09.981131 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:09.981181 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 17:02:09.981274 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 17:02:09.981384 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":64,"threads_started":1}
I20260808 17:02:09.982344 9622 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.194.61:42613: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c163d97601c94054ab26aacaf129394a"
member_type: NON_VOTER
last_known_addr {
host: "127.30.194.61"
port: 42613
}
attrs {
promote: true
}
}
dest_uuid: "b46b984f705b49c19aec28869ecc7594"
cas_config_opid_index: -1
I20260808 17:02:09.982632 9622 consensus_queue.cc:237] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } }
I20260808 17:02:09.983070 10500 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c163d97601c94054ab26aacaf129394a (127.30.194.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } } }
I20260808 17:02:09.983343 10408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:09.984127 10500 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.194.61:42613
I20260808 17:02:09.984163 10408 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Refusing update from remote peer b46b984f705b49c19aec28869ecc7594: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 17:02:09.984218 10500 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } } }
I20260808 17:02:09.984287 10500 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:09.984493 10348 master.cc:537] Master@127.30.194.61:42613 shutting down...
I20260808 17:02:09.984627 10499 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 17:02:09.985351 9596 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 17:02:09.985764 9596 consensus_peers.cc:487] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 -> Peer c163d97601c94054ab26aacaf129394a (127.30.194.61:42613): Couldn't send request to peer c163d97601c94054ab26aacaf129394a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 17:02:09.987107 10348 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:09.987190 10348 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:09.987241 10348 tablet_replica.cc:330] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: stopping tablet replica
I20260808 17:02:09.999536 10348 master.cc:552] Master@127.30.194.61:42613 shutdown complete.
I20260808 17:02:09.999573 10348 master_runner.cc:305] Clearing existing system tablet
I20260808 17:02:09.999681 10348 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:09.999719 10348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:10.000233 10348 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:10.000622 10507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.000765 10348 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:10.000818 10348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.000839 10348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal
uuid: "c163d97601c94054ab26aacaf129394a"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "63ae114d17e4b2dae54344d407dcd358"
server_key_iv: "efd350670040f14451363dc490cf3427"
server_key_version: "encryptionkey@0"
I20260808 17:02:10.001513 10348 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:10.002818 10348 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 17:02:10.002880 10348 log.cc:1196] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 17:02:10.003063 10348 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Deleting consensus metadata
I20260808 17:02:10.003159 10348 master_runner.cc:315] Copying system tablet from 127.30.194.62:35661
I20260808 17:02:10.003717 10348 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: tablet copy: Beginning tablet copy session from remote peer at address 127.30.194.62:35661
I20260808 17:02:10.009171 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.003945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:55634)
0808 17:02:10.004080 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.004089 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.004146 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.004288 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.004289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.004438 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.004487 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.004488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.005102 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.005103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.005623 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.005625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.005959 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.005960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.006023 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.006419 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.006422 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.006681 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.008070 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.008071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.008071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.008274 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.008511 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.008512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.008512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.008586 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.008827 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.008828 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.008885 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.008947 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.009047 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260808 17:02:10.009187 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.003994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55634 (local address 127.30.194.62:35661)
0808 17:02:10.004034 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.004036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.004142 (+ 106us) server_negotiation.cc:407] Connection header received
0808 17:02:10.004201 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.004201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.004211 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.004255 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.004554 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.005016 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.005695 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.005891 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.005958 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.006765 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.006766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.006767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.006774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.007984 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.008385 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.008386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.008386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.008444 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.008641 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.008642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.008642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.008775 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.008835 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.008982 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.009046 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:02:10.009783 9652 tablet_copy_service.cc:140] P b46b984f705b49c19aec28869ecc7594: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c163d97601c94054ab26aacaf129394a ({username='oryx', principal='oryx/127.30.194.61@KRBTEST.COM'} at 127.0.0.1:55634)
I20260808 17:02:10.009878 9652 tablet_copy_service.cc:161] P b46b984f705b49c19aec28869ecc7594: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c163d97601c94054ab26aacaf129394a at {username='oryx', principal='oryx/127.30.194.61@KRBTEST.COM'} at 127.0.0.1:55634: session id = c163d97601c94054ab26aacaf129394a-00000000000000000000000000000000
I20260808 17:02:10.010634 9652 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 17:02:10.011179 10348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 17:02:10.012006 10348 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: tablet copy: Starting download of 0 data blocks...
I20260808 17:02:10.012130 10348 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: tablet copy: Starting download of 1 WAL segments...
I20260808 17:02:10.012871 10348 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 17:02:10.013664 9652 tablet_copy_service.cc:342] P b46b984f705b49c19aec28869ecc7594: Request end of tablet copy session c163d97601c94054ab26aacaf129394a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.194.61@KRBTEST.COM'} at 127.0.0.1:55634
I20260808 17:02:10.013729 9652 tablet_copy_service.cc:433] P b46b984f705b49c19aec28869ecc7594: ending tablet copy session c163d97601c94054ab26aacaf129394a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c163d97601c94054ab26aacaf129394a
I20260808 17:02:10.016024 10348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:10.017026 10518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:10.017159 10524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:10.017241 10520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:10.017371 10348 server_base.cc:1034] running on GCE node
I20260808 17:02:10.017448 10348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:10.017567 10348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:10.018684 10348 hybrid_clock.cc:648] HybridClock initialized: now 1786208530018676 us; error 22 us; skew 500 ppm
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:10.019748 10348 init.cc:377] Logged in from keytab as oryx/127.30.194.61@KRBTEST.COM (short username oryx)
I20260808 17:02:10.019977 10348 webserver.cc:460] Webserver started at http://127.30.194.61:43905/ using document root <none> and password file <none>
I20260808 17:02:10.020058 10348 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:10.020080 10348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:10.020644 10348 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.021130 10532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.021251 10348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.021303 10348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal
uuid: "c163d97601c94054ab26aacaf129394a"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "63ae114d17e4b2dae54344d407dcd358"
server_key_iv: "efd350670040f14451363dc490cf3427"
server_key_version: "encryptionkey@0"
I20260808 17:02:10.021343 10348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:10.029505 10348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:10.030822 10348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:10.033639 10348 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.61:42613
I20260808 17:02:10.033888 10622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.61:42613 every 8 connection(s)
I20260808 17:02:10.033960 10348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
I20260808 17:02:10.034732 10626 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:02:10.035228 10626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Bootstrap starting.
I20260808 17:02:10.038043 10626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:02:10.038280 10626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Bootstrap complete.
I20260808 17:02:10.038475 10626 raft_consensus.cc:348] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } }
I20260808 17:02:10.038555 10626 raft_consensus.cc:729] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c163d97601c94054ab26aacaf129394a, State: Initialized, Role: LEARNER
I20260808 17:02:10.038600 10626 consensus_queue.cc:260] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } }
I20260808 17:02:10.038789 10626 sys_catalog.cc:564] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:10.038786 10629 sys_catalog.cc:455] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } } }
I20260808 17:02:10.038916 10629 sys_catalog.cc:458] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: This master's current role is: LEARNER
I20260808 17:02:10.040324 10644 catalog_manager.cc:1259] Loaded cluster ID: d4aac8015e2d4470b35e4009d8309005
I20260808 17:02:10.040416 10644 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: loading cluster ID for follower catalog manager: success
I20260808 17:02:10.041378 10644 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: acquiring CA information for follower catalog manager: success
I20260808 17:02:10.041617 10644 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:02:10.060642 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.054052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59960 (local address 127.30.194.61:42613)
0808 17:02:10.054161 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.054163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.054212 (+ 49us) server_negotiation.cc:407] Connection header received
0808 17:02:10.054235 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.054236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.054246 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.054298 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.055005 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.055670 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.056679 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.056870 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.056923 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.058300 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.058301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.058301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.058307 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.059620 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.060000 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.060001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.060001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.060052 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.060219 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.060220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.060220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.060328 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.060398 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.060515 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.060571 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for oryx/127.30.194.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.60.keytab.
Entry for principal oryx/127.30.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.60.keytab.
I20260808 17:02:10.085351 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.60:34603
--webserver_interface=127.30.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.60.keytab
--principal=oryx/127.30.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:35661,127.30.194.61:42613,127.30.194.60:34603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:10.168886 10652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:10.169107 10652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:10.169142 10652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:10.170288 10652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:10.170331 10652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:10.170353 10652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:10.170372 10652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:10.170392 10652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:10.171862 10652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:35661,127.30.194.61:42613,127.30.194.60:34603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.60.keytab
--principal=oryx/127.30.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.60:34603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:10.172099 10652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:10.172310 10652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:10.176918 10665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:10.176942 10663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:10.176924 10661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:10.177218 10652 server_base.cc:1034] running on GCE node
I20260808 17:02:10.177431 10652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:10.177659 10652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:10.178798 10652 hybrid_clock.cc:648] HybridClock initialized: now 1786208530178783 us; error 28 us; skew 500 ppm
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:10.181010 10652 init.cc:377] Logged in from keytab as oryx/127.30.194.60@KRBTEST.COM (short username oryx)
I20260808 17:02:10.181406 10652 webserver.cc:460] Webserver started at http://127.30.194.60:37703/ using document root <none> and password file <none>
I20260808 17:02:10.181587 10652 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:10.181654 10652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:10.181792 10652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:10.182752 10652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data/instance:
uuid: "498bf9fcca2f4ea5981c96dcad55d4c0"
format_stamp: "Formatted at 2026-08-08 17:02:10 on dist-test-slave-2j7r"
server_key: "55eb4aece04052eb20ef790119fb7bb7"
server_key_iv: "f0aa417b7017ba79b89827c89f587279"
server_key_version: "encryptionkey@0"
I20260808 17:02:10.183069 10652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal/instance:
uuid: "498bf9fcca2f4ea5981c96dcad55d4c0"
format_stamp: "Formatted at 2026-08-08 17:02:10 on dist-test-slave-2j7r"
server_key: "55eb4aece04052eb20ef790119fb7bb7"
server_key_iv: "f0aa417b7017ba79b89827c89f587279"
server_key_version: "encryptionkey@0"
I20260808 17:02:10.184301 10652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:02:10.184939 10677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.185103 10652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:10.185197 10652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal
uuid: "498bf9fcca2f4ea5981c96dcad55d4c0"
format_stamp: "Formatted at 2026-08-08 17:02:10 on dist-test-slave-2j7r"
server_key: "55eb4aece04052eb20ef790119fb7bb7"
server_key_iv: "f0aa417b7017ba79b89827c89f587279"
server_key_version: "encryptionkey@0"
I20260808 17:02:10.185259 10652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:10.203119 10652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:10.204859 10652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:10.204979 10652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:10.208925 10652 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.60:34603
I20260808 17:02:10.208963 10764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.60:34603 every 8 connection(s)
I20260808 17:02:10.209403 10652 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
I20260808 17:02:10.210210 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 10652
I20260808 17:02:10.210364 10766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:10.212596 10766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.194.60@KRBTEST.COM
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.60@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:10.221027 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.211109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47998 (local address 127.30.194.60:34603)
0808 17:02:10.211272 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.211283 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.211316 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:02:10.211886 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.211896 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.211961 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.212112 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.212706 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.213998 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.214731 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.215122 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.215196 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.217904 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.217911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.217911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.217917 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.219557 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.219945 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.219947 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.219948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.220051 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.220279 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.220280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.220281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.220512 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.220644 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.220754 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.220867 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260808 17:02:10.221231 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.213407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55644 (local address 127.30.194.62:35661)
0808 17:02:10.213451 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.213452 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.213461 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:10.213523 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.213524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.213534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.213588 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.214077 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.214558 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.215419 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.215709 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.215767 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.218558 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.218560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.218560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.218569 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.219963 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.220417 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.220418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.220419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.220484 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.220674 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.220675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.220676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.220795 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.220858 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.221044 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.221141 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:02:10.221413 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.213233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:55644)
0808 17:02:10.213380 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.213418 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.213476 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.213613 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.213615 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.213896 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.213991 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.213995 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.214649 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.214650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.215308 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.215311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.215805 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.215806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.215876 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.216384 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.216394 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.218466 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.220042 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.220049 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.220051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.220347 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.220544 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.220545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.220545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.220609 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.220860 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.220867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.220920 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.220999 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.221315 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260808 17:02:10.221939 10766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.60@KRBTEST.COM for oryx/127.30.194.61@KRBTEST.COM
I20260808 17:02:10.227398 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.222170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42613 (local address 127.0.0.1:59974)
0808 17:02:10.222228 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.222237 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.222307 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.222450 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.222451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.222568 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.222612 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.222613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.223221 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.223222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.223707 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.223709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.224060 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.224061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.224112 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.224450 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.224455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.225533 (+ 1078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.226717 (+ 1184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.226718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.226718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.226861 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.226994 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.226994 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.226994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.227035 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.227186 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.227187 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.227238 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.227281 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.227342 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 17:02:10.227417 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.222255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59974 (local address 127.30.194.61:42613)
0808 17:02:10.222300 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.222302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.222312 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:10.222362 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.222363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.222378 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.222446 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.222665 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.223133 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.223794 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.223991 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.224046 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.225617 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.225618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.225619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.225625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.226658 (+ 1033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.226913 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.226914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.226914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.226955 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.227066 (+ 111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.227067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.227067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.227150 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.227191 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.227298 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.227359 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:02:10.227802 10766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.60@KRBTEST.COM for oryx/127.30.194.60@KRBTEST.COM
I20260808 17:02:10.232774 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.228005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:34603 (local address 127.0.0.1:48006)
0808 17:02:10.228042 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.228052 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.228083 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.228166 (+ 83us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.228166 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.228285 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.228309 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.228310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.228835 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.228836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.229318 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.229320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.229604 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.229604 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.229635 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.229936 (+ 301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.229940 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.230900 (+ 960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.231998 (+ 1098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.231999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.231999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.232143 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.232325 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.232326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.232326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.232399 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.232568 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.232568 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.232599 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.232642 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.232715 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 17:02:10.232841 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.228033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48006 (local address 127.30.194.60:34603)
0808 17:02:10.228075 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.228076 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.228084 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:10.228117 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.228118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.228124 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.228163 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.228352 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.228780 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.229374 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.229556 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.229601 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.230966 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.230968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.230968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.230973 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.231928 (+ 955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.232205 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.232205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.232205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.232245 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.232450 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.232451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.232451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.232528 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.232568 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.232657 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.232762 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":25}
I20260808 17:02:10.234000 10766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: Bootstrap starting.
I20260808 17:02:10.234793 10766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:10.235042 10766 log.cc:824] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:10.235688 10766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: No bootstrap required, opened a new log
I20260808 17:02:10.237054 10766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } }
I20260808 17:02:10.237202 10766 raft_consensus.cc:374] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:10.237257 10766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498bf9fcca2f4ea5981c96dcad55d4c0, State: Initialized, Role: FOLLOWER
I20260808 17:02:10.237380 10766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } }
I20260808 17:02:10.237627 10775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } } }
I20260808 17:02:10.237682 10766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:10.237707 10775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 17:02:10.239351 10789 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:02:10.239456 10789 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:02:10.241324 10652 master_runner.cc:428] Detected that this master 498bf9fcca2f4ea5981c96dcad55d4c0 is joining an existing cluster
I20260808 17:02:10.241376 10652 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.194.60:34603
I20260808 17:02:10.249050 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.242789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59988 (local address 127.30.194.61:42613)
0808 17:02:10.242832 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.242834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.242840 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:02:10.242857 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.242858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.242866 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.242917 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.243396 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.244109 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.245219 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.245506 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.245547 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.246835 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.246836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.246837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.246842 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.248080 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.248468 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.248469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.248469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.248511 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.248660 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.248660 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.248661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.248744 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.248785 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.248942 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.248994 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:02:10.249143 10799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.242412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42613 (local address 127.0.0.1:59988)
0808 17:02:10.242554 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.242562 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.242601 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.242929 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.242930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.243235 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.243298 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.243301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.244205 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.244206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.245121 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.245124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.245620 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.245620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.245992 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.246322 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.246326 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.246712 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.248156 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.248157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.248157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.248396 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.248550 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.248551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.248551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.248611 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.248798 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.248799 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.248852 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.248927 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.249019 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":101,"threads_started":1}
I20260808 17:02:10.267853 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.242614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:34603 (local address 127.0.0.1:48010)
0808 17:02:10.242708 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.242716 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.242750 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.242958 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.242959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.243265 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.243319 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.243321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.244102 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.244103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.244977 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.244979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.245488 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.245489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.245543 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.245916 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.245920 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.265356 (+ 19436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.266849 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.266850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.266850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.267055 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.267265 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.267266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.267266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.267324 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.267549 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.267549 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.267586 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.267644 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.267756 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":34,"thread_start_us":60,"threads_started":1}
I20260808 17:02:10.268016 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.242840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.30.194.60:34603)
0808 17:02:10.242873 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.242874 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.242880 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:02:10.242892 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.242893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.242899 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.242941 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.243392 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.244032 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.245065 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.245373 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.245412 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.265446 (+ 20034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.265447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.265448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.265454 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.266757 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.267135 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.267136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.267136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.267190 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.267381 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.267382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.267382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.267498 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.267556 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.267671 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.267945 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":203,"server-negotiator.queue_time_us":17}
I20260808 17:02:10.268538 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.242400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55646 (local address 127.30.194.62:35661)
0808 17:02:10.242441 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.242443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.242569 (+ 126us) server_negotiation.cc:407] Connection header received
0808 17:02:10.242661 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.242662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.242671 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.242699 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.243218 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.243963 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.245069 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.245377 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.245412 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.265923 (+ 20511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.265925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.265925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.265943 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.267406 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.267765 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.267766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.267766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.267822 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.268016 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.268017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.268018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.268146 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.268188 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.268371 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.268451 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:02:10.268541 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.242359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:55646)
0808 17:02:10.242507 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.242516 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.242593 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.242774 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.242775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.243078 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.243131 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.243133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.244049 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.244049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.244984 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.244986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.245489 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.245490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.245543 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.265522 (+ 19979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.265526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.265835 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.267470 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.267470 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.267471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.267660 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.267884 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.267884 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.267884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.267942 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.268233 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.268234 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.268274 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.268338 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.268435 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":40,"thread_start_us":73,"threads_started":1}
I20260808 17:02:10.272508 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 9725
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208529, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.62@KRBTEST.COM for oryx/127.30.194.60@KRBTEST.COM
I20260808 17:02:10.296277 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.270288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:34603 (local address 127.0.0.1:48012)
0808 17:02:10.270333 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.270342 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.270377 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.270519 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.270520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.270648 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.270682 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.270683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.271248 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.271249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.271736 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.271737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.272121 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.272122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.272199 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.292411 (+ 20212us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.292416 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.293730 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.295107 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.295108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.295108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.295303 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.295481 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.295482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.295482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.295552 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.295895 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.295897 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.295954 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.296033 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.296156 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 17:02:10.296862 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.270374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48012 (local address 127.30.194.60:34603)
0808 17:02:10.270422 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.270424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.270431 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:10.270457 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.270458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.270466 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.270509 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.270747 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.271181 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.271800 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.272042 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.272095 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.293819 (+ 21724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.293821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.293821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.293827 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.295032 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.295372 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.295373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.295374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.295424 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.295634 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.295635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.295635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.295789 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.295860 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.296709 (+ 849us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.296783 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:02:10.297544 9622 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.194.60:34603: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0"
member_type: NON_VOTER
last_known_addr {
host: "127.30.194.60"
port: 34603
}
attrs {
promote: true
}
}
dest_uuid: "b46b984f705b49c19aec28869ecc7594"
cas_config_opid_index: 5
I20260808 17:02:10.297784 9622 consensus_queue.cc:237] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } }
I20260808 17:02:10.298282 10500 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.299194 10500 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.194.60:34603
I20260808 17:02:10.299270 10500 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.299338 10500 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:10.299638 10652 master.cc:537] Master@127.30.194.60:34603 shutting down...
I20260808 17:02:10.299651 10728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:10.300585 10728 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 FOLLOWER]: Refusing update from remote peer b46b984f705b49c19aec28869ecc7594: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 17:02:10.301221 10499 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 17:02:10.301959 9596 consensus_peers.cc:487] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 -> Peer 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60:34603): Couldn't send request to peer 498bf9fcca2f4ea5981c96dcad55d4c0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 17:02:10.302050 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:37889
--local_ip_for_outbound_sockets=127.30.194.1
--tserver_master_addrs=127.30.194.62:35661
--webserver_port=44543
--webserver_interface=127.30.194.1
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab
--principal=oryx/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.194.62:35661,127.30.194.61:42613
--tserver_master_addrs=127.30.194.62:35661,127.30.194.61:42613,127.30.194.60:34603 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:02:10.303261 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.298244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42613 (local address 127.0.0.1:59990)
0808 17:02:10.299815 (+ 1571us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.299827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.299875 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.300078 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.300079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.300081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:10.300116 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.300117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.301014 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.301015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.302217 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.302221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.302994 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.302995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.303063 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.303086 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.303157 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.303192 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1555}
I20260808 17:02:10.303561 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.298320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59990 (local address 127.30.194.61:42613)
0808 17:02:10.298367 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.298368 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.299926 (+ 1558us) server_negotiation.cc:407] Connection header received
0808 17:02:10.299953 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.299954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.299976 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.300028 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:10.300178 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.300923 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.302318 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.302908 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.302982 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.303055 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.303457 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.303499 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:02:10.303954 10576 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 LEARNER]: Refusing update from remote peer b46b984f705b49c19aec28869ecc7594: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 17:02:10.304365 10499 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 17:02:10.304917 10576 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.306185 10652 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:10.306277 10652 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:10.306270 10499 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594: attempting to promote NON_VOTER c163d97601c94054ab26aacaf129394a to VOTER
I20260808 17:02:10.306319 10652 tablet_replica.cc:330] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: stopping tablet replica
I20260808 17:02:10.306322 10629 sys_catalog.cc:455] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b46b984f705b49c19aec28869ecc7594. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.306397 10629 sys_catalog.cc:458] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: This master's current role is: LEARNER
I20260808 17:02:10.306430 10629 sys_catalog.cc:455] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: NON_VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: true } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.306466 10629 sys_catalog.cc:458] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: This master's current role is: LEARNER
I20260808 17:02:10.306456 10499 consensus_queue.cc:237] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } }
I20260808 17:02:10.307219 10576 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 LEARNER]: Refusing update from remote peer b46b984f705b49c19aec28869ecc7594: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260808 17:02:10.307449 10679 connection.cc:441] server connection from 127.0.0.1:48012 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48012 (request call id 5) could send its response
I20260808 17:02:10.307480 10500 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260808 17:02:10.307746 9596 consensus_peers.cc:487] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 -> Peer 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60:34603): Couldn't send request to peer 498bf9fcca2f4ea5981c96dcad55d4c0. Status: Network error: recv got EOF from 127.30.194.60:34603 (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260808 17:02:10.308534 10500 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c163d97601c94054ab26aacaf129394a (127.30.194.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.309191 10500 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.309271 10500 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:10.309260 10499 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.309324 10499 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:10.309538 10576 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c163d97601c94054ab26aacaf129394a (127.30.194.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.310276 10804 sys_catalog.cc:455] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.310356 10804 sys_catalog.cc:458] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:02:10.319345 10652 master.cc:552] Master@127.30.194.60:34603 shutdown complete.
I20260808 17:02:10.319383 10652 master_runner.cc:305] Clearing existing system tablet
I20260808 17:02:10.319494 10652 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:10.319536 10652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:10.320060 10652 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:10.320472 10809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.320592 10652 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:10.320645 10652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.320669 10652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal
uuid: "498bf9fcca2f4ea5981c96dcad55d4c0"
format_stamp: "Formatted at 2026-08-08 17:02:10 on dist-test-slave-2j7r"
server_key: "55eb4aece04052eb20ef790119fb7bb7"
server_key_iv: "f0aa417b7017ba79b89827c89f587279"
server_key_version: "encryptionkey@0"
I20260808 17:02:10.321288 10652 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:10.322432 10652 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 17:02:10.322490 10652 log.cc:1196] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 17:02:10.322675 10652 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: Deleting consensus metadata
I20260808 17:02:10.322794 10652 master_runner.cc:315] Copying system tablet from 127.30.194.62:35661
I20260808 17:02:10.323310 10652 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.194.62:35661
I20260808 17:02:10.328466 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.323604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55656 (local address 127.30.194.62:35661)
0808 17:02:10.323641 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.323643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.323721 (+ 78us) server_negotiation.cc:407] Connection header received
0808 17:02:10.323780 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.323780 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.323789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.323841 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.324098 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.324560 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.325243 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.325433 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.325482 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.326329 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.326331 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.326331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.326338 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.327517 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.327807 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.327808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.327809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.327864 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.328019 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.328020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.328020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.328119 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.328173 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.328309 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.328380 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:02:10.328471 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.323559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:55656)
0808 17:02:10.323665 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.323674 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.323726 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.323845 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.323846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.324000 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.324033 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.324034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.324626 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.324627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.325167 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.325169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.325496 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.325497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.325550 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.325931 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.325945 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.326256 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.327588 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.327589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.327590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.327749 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.327914 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.327915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.327915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.327970 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.328170 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.328171 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.328223 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.328281 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.328379 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260808 17:02:10.328975 9652 tablet_copy_service.cc:140] P b46b984f705b49c19aec28869ecc7594: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 498bf9fcca2f4ea5981c96dcad55d4c0 ({username='oryx', principal='oryx/127.30.194.60@KRBTEST.COM'} at 127.0.0.1:55656)
I20260808 17:02:10.329028 9652 tablet_copy_service.cc:161] P b46b984f705b49c19aec28869ecc7594: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 498bf9fcca2f4ea5981c96dcad55d4c0 at {username='oryx', principal='oryx/127.30.194.60@KRBTEST.COM'} at 127.0.0.1:55656: session id = 498bf9fcca2f4ea5981c96dcad55d4c0-00000000000000000000000000000000
I20260808 17:02:10.329459 9652 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 17:02:10.329849 10652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 17:02:10.330616 10652 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: tablet copy: Starting download of 0 data blocks...
I20260808 17:02:10.330749 10652 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: tablet copy: Starting download of 1 WAL segments...
I20260808 17:02:10.331341 10652 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 17:02:10.331995 9652 tablet_copy_service.cc:342] P b46b984f705b49c19aec28869ecc7594: Request end of tablet copy session 498bf9fcca2f4ea5981c96dcad55d4c0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.194.60@KRBTEST.COM'} at 127.0.0.1:55656
I20260808 17:02:10.332044 9652 tablet_copy_service.cc:433] P b46b984f705b49c19aec28869ecc7594: ending tablet copy session 498bf9fcca2f4ea5981c96dcad55d4c0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 498bf9fcca2f4ea5981c96dcad55d4c0
I20260808 17:02:10.334165 10652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:10.335152 10819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:10.335218 10827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:10.335592 10652 server_base.cc:1034] running on GCE node
W20260808 17:02:10.337227 10822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:10.337442 10652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:10.337592 10652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:10.338747 10652 hybrid_clock.cc:648] HybridClock initialized: now 1786208530338735 us; error 27 us; skew 500 ppm
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:10.339891 10652 init.cc:377] Logged in from keytab as oryx/127.30.194.60@KRBTEST.COM (short username oryx)
I20260808 17:02:10.340116 10652 webserver.cc:460] Webserver started at http://127.30.194.60:37673/ using document root <none> and password file <none>
I20260808 17:02:10.340233 10652 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:10.340266 10652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:10.340821 10652 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:10.341326 10836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.341470 10652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.341512 10652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal
uuid: "498bf9fcca2f4ea5981c96dcad55d4c0"
format_stamp: "Formatted at 2026-08-08 17:02:10 on dist-test-slave-2j7r"
server_key: "55eb4aece04052eb20ef790119fb7bb7"
server_key_iv: "f0aa417b7017ba79b89827c89f587279"
server_key_version: "encryptionkey@0"
I20260808 17:02:10.341562 10652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:10.345854 10652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:10.347257 10652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:10.349983 10652 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.60:34603
I20260808 17:02:10.350329 10652 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
I20260808 17:02:10.350550 10920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.60:34603 every 8 connection(s)
I20260808 17:02:10.350996 10921 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:02:10.351339 10921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: Bootstrap starting.
I20260808 17:02:10.353452 10921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:02:10.353631 10921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: Bootstrap complete.
I20260808 17:02:10.353808 10921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } }
I20260808 17:02:10.353886 10921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 498bf9fcca2f4ea5981c96dcad55d4c0, State: Initialized, Role: LEARNER
I20260808 17:02:10.353921 10921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } }
I20260808 17:02:10.354085 10921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:10.355333 10938 catalog_manager.cc:1259] Loaded cluster ID: d4aac8015e2d4470b35e4009d8309005
I20260808 17:02:10.355408 10938 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: loading cluster ID for follower catalog manager: success
I20260808 17:02:10.355193 10924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.355515 10924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: This master's current role is: LEARNER
I20260808 17:02:10.356463 10938 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: acquiring CA information for follower catalog manager: success
I20260808 17:02:10.356762 10938 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 17:02:10.417062 10801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:10.417254 10801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:10.417276 10801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:10.418351 10801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:10.418385 10801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:10.418408 10801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:10.419802 10801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab
--principal=oryx/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:37889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=44543
--tserver_master_addrs=127.30.194.62:35661,127.30.194.61:42613,127.30.194.60:34603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:10.419971 10801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:10.420142 10801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:10.424944 10951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:10.424953 10948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:10.424963 10947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:10.425028 10801 server_base.cc:1034] running on GCE node
I20260808 17:02:10.425401 10801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:10.425625 10801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:10.426769 10801 hybrid_clock.cc:648] HybridClock initialized: now 1786208530426753 us; error 27 us; skew 500 ppm
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:10.429052 10801 init.cc:377] Logged in from keytab as oryx/127.30.194.1@KRBTEST.COM (short username oryx)
I20260808 17:02:10.429476 10801 webserver.cc:460] Webserver started at http://127.30.194.1:44543/ using document root <none> and password file <none>
I20260808 17:02:10.429687 10801 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:10.429749 10801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:10.430940 10801 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:10.431434 10959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.431581 10801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:10.431664 10801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "b4b7243cd2ab4fd39a4a4968c110d86e"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "89ca2f1babd5ef292891a1515b3d5ed5"
server_key_iv: "e4eaf3a91ae34bca5cd131287538ea0a"
server_key_version: "encryptionkey@0"
I20260808 17:02:10.431942 10801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:10.442901 10801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:10.444571 10801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:10.444700 10801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:10.444895 10801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:10.445194 10801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:10.445228 10801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.445264 10801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:10.445289 10801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.450748 10801 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:37889
I20260808 17:02:10.450803 11128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:37889 every 8 connection(s)
I20260808 17:02:10.451089 10801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.60@KRBTEST.COM
I20260808 17:02:10.457779 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 10801
I20260808 17:02:10.457927 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 9918
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.61@KRBTEST.COM
I20260808 17:02:10.470000 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.451730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:49701 (local address 127.30.194.62:35661)
0808 17:02:10.451777 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.451779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.452007 (+ 228us) server_negotiation.cc:407] Connection header received
0808 17:02:10.452528 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.452529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.452542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.452571 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.453450 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.454184 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.455396 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.455604 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.455659 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.467185 (+ 11526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.467187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.467187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.467197 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.468568 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.468995 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.468996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.468997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.469060 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.469338 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.469339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.469340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.469464 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.469529 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.469815 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.469901 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:02:10.470156 11138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.451730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.1:49701)
0808 17:02:10.451891 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.451920 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.452423 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.452649 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.452654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.453199 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.453331 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.453337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.454268 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.454269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.455326 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.455329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.455673 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.455674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.455814 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.456400 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.456404 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.467061 (+ 10657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.468657 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.468666 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.468669 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.468916 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.469147 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.469148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.469149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.469274 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.469527 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.469535 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.469676 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.469768 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.469936 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":1,"spinlock_wait_cycles":5888,"thread_start_us":79,"threads_started":1}
I20260808 17:02:10.470618 11133 heartbeater.cc:347] Connected to a master server at 127.30.194.62:35661
I20260808 17:02:10.470734 11133 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:10.470927 11133 heartbeater.cc:511] Master 127.30.194.62:35661 requested a full tablet report, sending...
I20260808 17:02:10.471426 9622 ts_manager.cc:194] Re-registered known tserver with Master: b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889)
I20260808 17:02:10.471818 9622 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:49701
I20260808 17:02:10.478156 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.451730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:34603 (local address 127.30.194.1:39861)
0808 17:02:10.451882 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.451920 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.452414 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.452699 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.452700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.453043 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.453330 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.453336 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.454614 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.454615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.455312 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.455315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.455809 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.455810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.455885 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.456449 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.456453 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.475129 (+ 18676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.477028 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.477029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.477030 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.477306 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.477522 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.477523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.477523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.477576 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.477833 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.477835 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.477878 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.477945 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.478048 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":5,"thread_start_us":91,"threads_started":1}
I20260808 17:02:10.478163 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.451951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:39861 (local address 127.30.194.60:34603)
0808 17:02:10.452368 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.452370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.452377 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:10.452622 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.452623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.452634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.452683 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.453418 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.454551 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.455405 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.455701 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.455754 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.475256 (+ 19502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.475258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.475258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.475269 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.476924 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.477396 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.477398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.477398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.477461 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.477653 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.477654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.477654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.477775 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.477833 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.477993 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.478075 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":63,"threads_started":1}
I20260808 17:02:10.479017 11129 heartbeater.cc:347] Connected to a master server at 127.30.194.60:34603
I20260808 17:02:10.479077 11129 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:10.479159 11129 heartbeater.cc:511] Master 127.30.194.60:34603 requested a full tablet report, sending...
I20260808 17:02:10.479501 10857 ts_manager.cc:194] Registered new tserver with Master: b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889)
I20260808 17:02:10.480783 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.452136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:39693 (local address 127.30.194.61:42613)
0808 17:02:10.452205 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.452206 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.452216 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:10.452512 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.452514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.452526 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.452571 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.453439 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.454000 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.455431 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.455729 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.455783 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.478122 (+ 22339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.478124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.478124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.478133 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.479602 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.479977 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.479978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.479978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.480039 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.480282 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.480283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.480284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.480423 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.480486 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.480646 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.480713 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 17:02:10.480885 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.451993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42613 (local address 127.30.194.1:39693)
0808 17:02:10.452143 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.452151 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.452416 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.452650 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.452654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.453057 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.453331 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.453337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.454118 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.454119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.455355 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.455356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.455796 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.455797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.455859 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.456345 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.456354 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.478016 (+ 21662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.479696 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.479697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.479697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.479907 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.480105 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.480106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.480106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.480180 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.480506 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.480507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.480554 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.480621 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.480733 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":72,"thread_start_us":56,"threads_started":1}
I20260808 17:02:10.481274 11134 heartbeater.cc:347] Connected to a master server at 127.30.194.61:42613
I20260808 17:02:10.481324 11134 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:10.481369 11134 heartbeater.cc:511] Master 127.30.194.61:42613 requested a full tablet report, sending...
I20260808 17:02:10.481686 10558 ts_manager.cc:194] Registered new tserver with Master: b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889)
I20260808 17:02:10.482894 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:35543
--local_ip_for_outbound_sockets=127.30.194.2
--tserver_master_addrs=127.30.194.62:35661
--webserver_port=45083
--webserver_interface=127.30.194.2
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab
--principal=oryx/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.194.62:35661,127.30.194.61:42613
--tserver_master_addrs=127.30.194.62:35661,127.30.194.61:42613,127.30.194.60:34603 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:10.563598 11142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:10.563791 11142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:10.563822 11142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:10.564894 11142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:10.564931 11142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:10.564965 11142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:10.566407 11142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab
--principal=oryx/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:35543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=45083
--tserver_master_addrs=127.30.194.62:35661,127.30.194.61:42613,127.30.194.60:34603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:10.566624 11142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:10.566821 11142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:10.571416 11151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:10.571470 11154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:10.571647 11142 server_base.cc:1034] running on GCE node
W20260808 17:02:10.571429 11152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:10.571885 11142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:10.572072 11142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:10.573201 11142 hybrid_clock.cc:648] HybridClock initialized: now 1786208530573187 us; error 32 us; skew 500 ppm
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:10.575558 11142 init.cc:377] Logged in from keytab as oryx/127.30.194.2@KRBTEST.COM (short username oryx)
I20260808 17:02:10.576031 11142 webserver.cc:460] Webserver started at http://127.30.194.2:45083/ using document root <none> and password file <none>
I20260808 17:02:10.576267 11142 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:10.576339 11142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:10.577759 11142 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:10.578310 11164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.578450 11142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:10.578537 11142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "925364577a1542148cb6c46abf5ab9ec"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "9681f678dbb53fe2b100d1490e2c4748"
server_key_iv: "d4310a9c26e3b3d77873610b06da750b"
server_key_version: "encryptionkey@0"
I20260808 17:02:10.578837 11142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:10.585453 11142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:10.587533 11142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:10.587658 11142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:10.587858 11142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:10.588130 11142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:10.588166 11142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.588196 11142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:10.588225 11142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.594161 11142 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:35543
I20260808 17:02:10.594214 11334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:35543 every 8 connection(s)
I20260808 17:02:10.594511 11142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:02:10.597596 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 11142
I20260808 17:02:10.597767 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 10135
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.61@KRBTEST.COM
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.60@KRBTEST.COM
I20260808 17:02:10.608705 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.595155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:60613 (local address 127.30.194.62:35661)
0808 17:02:10.595201 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.595203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.595475 (+ 272us) server_negotiation.cc:407] Connection header received
0808 17:02:10.596005 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.596006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.596020 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.596056 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.596891 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.597645 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.601100 (+ 3455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.601385 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.601426 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.605043 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.605044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.605044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.605052 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.607471 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.607803 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.607804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.607805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.607857 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.608039 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.608040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.608040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.608151 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.608188 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.608533 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.608611 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:02:10.608705 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.595553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:56031 (local address 127.30.194.60:34603)
0808 17:02:10.595606 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.595608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.595618 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:10.596004 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.596005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.596023 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.596056 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.596882 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.597447 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.601694 (+ 4247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.601900 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.601956 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.605018 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.605020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.605020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.605028 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.607225 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.607671 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.607672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.607673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.607740 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.607998 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.607999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.608000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.608124 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.608164 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.608532 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.608612 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:02:10.609213 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.595152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:34603 (local address 127.30.194.2:56031)
0808 17:02:10.595410 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.595419 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.595881 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.596138 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.596143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.596619 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.596763 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.596769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.597589 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.597590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.600317 (+ 2727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.600320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.601963 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.601964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.602023 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.602728 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.602734 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.604904 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.607313 (+ 2409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.607321 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.607323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.607592 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.607811 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.607811 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.607812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.607928 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.608211 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.608217 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.608366 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.608961 (+ 595us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.609037 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":73,"spinlock_wait_cycles":7040,"thread_start_us":95,"threads_started":1}
I20260808 17:02:10.609226 11345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.595152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.2:60613)
0808 17:02:10.595346 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.595388 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.595877 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.596138 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.596143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.596616 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.596763 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.596769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.597745 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.597747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.600995 (+ 3248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.600997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.601493 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.601494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.601542 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.602576 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.602581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.604942 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.607549 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.607550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.607550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.607723 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.607920 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.607920 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.607921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.607972 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.608240 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.608241 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.608366 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.608798 (+ 432us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.608970 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
I20260808 17:02:10.609383 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.595518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:43617 (local address 127.30.194.61:42613)
0808 17:02:10.595559 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.595562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.595690 (+ 128us) server_negotiation.cc:407] Connection header received
0808 17:02:10.595959 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.595960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.595972 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.596022 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.596853 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.597655 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.600675 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.600962 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.601019 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.605166 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.605167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.605168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.605177 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.607720 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.608251 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.608252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.608253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.608326 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.608617 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.608618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.608619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.608800 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.608860 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.609177 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.609277 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:02:10.609747 11350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.595433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42613 (local address 127.30.194.2:43617)
0808 17:02:10.595624 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.595633 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.595886 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.596346 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.596347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.596573 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.596761 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.596765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.597861 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.597862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.600605 (+ 2743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.600607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.601100 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.601101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.601232 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.602483 (+ 1251us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.602491 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.605017 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.607843 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.607844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.607845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.608136 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.608429 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.608430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.608430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.608511 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.608926 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.608927 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.608983 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.609486 (+ 503us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.609611 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":55,"threads_started":1}
I20260808 17:02:10.609779 11336 heartbeater.cc:347] Connected to a master server at 127.30.194.60:34603
I20260808 17:02:10.609865 11336 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:10.610015 11336 heartbeater.cc:511] Master 127.30.194.60:34603 requested a full tablet report, sending...
I20260808 17:02:10.610493 10857 ts_manager.cc:194] Registered new tserver with Master: 925364577a1542148cb6c46abf5ab9ec (127.30.194.2:35543)
I20260808 17:02:10.624918 11337 heartbeater.cc:347] Connected to a master server at 127.30.194.62:35661
I20260808 17:02:10.624986 11337 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:10.625075 11337 heartbeater.cc:511] Master 127.30.194.62:35661 requested a full tablet report, sending...
I20260808 17:02:10.625440 9622 ts_manager.cc:194] Re-registered known tserver with Master: 925364577a1542148cb6c46abf5ab9ec (127.30.194.2:35543)
I20260808 17:02:10.626084 9622 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:60613
I20260808 17:02:10.626271 11340 heartbeater.cc:347] Connected to a master server at 127.30.194.61:42613
I20260808 17:02:10.626338 11340 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:10.626431 11340 heartbeater.cc:511] Master 127.30.194.61:42613 requested a full tablet report, sending...
I20260808 17:02:10.626729 10558 ts_manager.cc:194] Registered new tserver with Master: 925364577a1542148cb6c46abf5ab9ec (127.30.194.2:35543)
I20260808 17:02:10.631763 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:33505
--local_ip_for_outbound_sockets=127.30.194.3
--tserver_master_addrs=127.30.194.62:35661
--webserver_port=43371
--webserver_interface=127.30.194.3
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab
--principal=oryx/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.194.62:35661,127.30.194.61:42613
--tserver_master_addrs=127.30.194.62:35661,127.30.194.61:42613,127.30.194.60:34603 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:02:10.693660 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.690573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:34603 (local address 127.0.0.1:48022)
0808 17:02:10.690622 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.690636 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.690687 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.690899 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.690900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.690901 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:10.690949 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.690950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.691897 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.691897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.692603 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.692605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.693439 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.693440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.693498 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.693503 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.693558 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.693597 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 17:02:10.693707 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.690663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48022 (local address 127.30.194.60:34603)
0808 17:02:10.690735 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.690738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.690748 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:10.690782 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.690783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.690810 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.690897 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:10.691009 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.691811 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.692693 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.693333 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.693422 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.693495 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.693575 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.693616 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:02:10.694020 10880 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 17:02:10.694162 10924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b46b984f705b49c19aec28869ecc7594. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.694315 10924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: This master's current role is: LEARNER
W20260808 17:02:10.714371 11351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:10.714556 11351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:10.714589 11351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:10.715692 11351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:10.715736 11351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:10.715775 11351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:10.717259 11351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab
--principal=oryx/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:33505
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=43371
--tserver_master_addrs=127.30.194.62:35661,127.30.194.61:42613,127.30.194.60:34603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:10.717479 11351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:10.717687 11351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:10.722345 11364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:10.722373 11361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:10.722374 11360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:10.722406 11351 server_base.cc:1034] running on GCE node
I20260808 17:02:10.722698 11351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:10.722923 11351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:10.724049 11351 hybrid_clock.cc:648] HybridClock initialized: now 1786208530724034 us; error 34 us; skew 500 ppm
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:10.726382 11351 init.cc:377] Logged in from keytab as oryx/127.30.194.3@KRBTEST.COM (short username oryx)
I20260808 17:02:10.726738 11351 webserver.cc:460] Webserver started at http://127.30.194.3:43371/ using document root <none> and password file <none>
I20260808 17:02:10.726915 11351 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:10.726962 11351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:10.728134 11351 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:02:10.728653 11371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.728806 11351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.728873 11351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "99d1b745ca0b424b920ed368038dc8d2"
format_stamp: "Formatted at 2026-08-08 17:02:09 on dist-test-slave-2j7r"
server_key: "987ff17db7fe3e915273051ee3d1e691"
server_key_iv: "ecc29a3e35b41900e94046b67d54d193"
server_key_version: "encryptionkey@0"
I20260808 17:02:10.729175 11351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:10.739426 11351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:10.741443 11351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:10.741564 11351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:10.741756 11351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:10.742044 11351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:10.742076 11351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.742101 11351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:10.742125 11351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:10.747836 11351 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:33505
I20260808 17:02:10.747893 11559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:33505 every 8 connection(s)
I20260808 17:02:10.748211 11351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.61@KRBTEST.COM
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:10.756626 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 11351
I20260808 17:02:10.758661 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.748938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42613 (local address 127.30.194.3:49279)
0808 17:02:10.749139 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.749181 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.749721 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.749982 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.749986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.750327 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.750547 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.750553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.751514 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.751515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.752561 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.752564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.753041 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.753042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.753192 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.753641 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.753650 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.755335 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.756898 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.756905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.756907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.757187 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.757550 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.757551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.757551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.757693 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.758006 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.758014 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.758171 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.758288 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.758437 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":37,"thread_start_us":100,"threads_started":1}
Aug 08 17:02:10 dist-test-slave-2j7r krb5kdc[9546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208530, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.60@KRBTEST.COM
I20260808 17:02:10.759065 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.749255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:49279 (local address 127.30.194.61:42613)
0808 17:02:10.749301 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.749303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.749313 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:10.749852 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.749854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.749868 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.749922 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.750665 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.751401 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.752671 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.752959 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.753024 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.755440 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.755442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.755442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.755450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.756822 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.757269 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.757270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.757271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.757334 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.757774 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.757775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.757775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.757918 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.757976 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.758906 (+ 930us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.758982 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:02:10.759462 11564 heartbeater.cc:347] Connected to a master server at 127.30.194.61:42613
I20260808 17:02:10.759554 11564 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:10.759719 11564 heartbeater.cc:511] Master 127.30.194.61:42613 requested a full tablet report, sending...
I20260808 17:02:10.760098 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.749259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:46059 (local address 127.30.194.62:35661)
0808 17:02:10.749349 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.749351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.749430 (+ 79us) server_negotiation.cc:407] Connection header received
0808 17:02:10.749969 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.749970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.749981 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.750022 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.750637 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.751496 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.752653 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.753022 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.753076 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.756060 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.756061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.756061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.756069 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.758638 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.759021 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.759022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.759022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.759078 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.759278 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.759279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.759280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.759408 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.759453 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.759942 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.760024 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 17:02:10.760142 10558 ts_manager.cc:194] Registered new tserver with Master: 99d1b745ca0b424b920ed368038dc8d2 (127.30.194.3:33505)
I20260808 17:02:10.760403 11568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.748938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.3:46059)
0808 17:02:10.749366 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.749375 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.749736 (+ 361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.750038 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.750039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.750320 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.750547 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.750551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.751552 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.751553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.752559 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.752562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.753140 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.753141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.753200 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.753805 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.753810 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.755949 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.758726 (+ 2777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.758727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.758728 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.758937 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.759150 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.759151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.759151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.759208 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.759502 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.759503 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.759817 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.760178 (+ 361us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.760273 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":59,"thread_start_us":127,"threads_started":1}
I20260808 17:02:10.760718 11563 heartbeater.cc:347] Connected to a master server at 127.30.194.62:35661
I20260808 17:02:10.760779 11563 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:10.760839 11563 heartbeater.cc:511] Master 127.30.194.62:35661 requested a full tablet report, sending...
I20260808 17:02:10.761130 9622 ts_manager.cc:194] Re-registered known tserver with Master: 99d1b745ca0b424b920ed368038dc8d2 (127.30.194.3:33505)
I20260808 17:02:10.762104 9622 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:46059
I20260808 17:02:10.762327 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.753328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:34603 (local address 127.30.194.3:40525)
0808 17:02:10.753445 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.753454 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.753521 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.755211 (+ 1690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.755211 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.755374 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:10.755427 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.755428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.756180 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.756181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.756967 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.756970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.757451 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.757452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.757517 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.757895 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:10.757899 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:10.759225 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:10.761162 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.761163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.761163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.761352 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.761523 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:10.761524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:10.761524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:10.761580 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:10.761836 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:10.761837 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:10.761875 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.761939 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.762029 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260808 17:02:10.762602 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.755030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:40525 (local address 127.30.194.60:34603)
0808 17:02:10.755084 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.755086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.755095 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:10.755120 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.755121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.755133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.755186 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.755528 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.756090 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.757053 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.757376 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.757437 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.759320 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.759321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.759321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.759328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.761103 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.761415 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.761416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.761417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.761471 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.761644 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.761644 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.761645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.761757 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.761810 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.762441 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.762516 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:02:10.762980 11562 heartbeater.cc:347] Connected to a master server at 127.30.194.60:34603
I20260808 17:02:10.763052 11562 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:10.763111 11562 heartbeater.cc:511] Master 127.30.194.60:34603 requested a full tablet report, sending...
I20260808 17:02:10.763394 10857 ts_manager.cc:194] Registered new tserver with Master: 99d1b745ca0b424b920ed368038dc8d2 (127.30.194.3:33505)
I20260808 17:02:10.767561 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.758400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.30.194.60:34603)
0808 17:02:10.759521 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.759523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.759838 (+ 315us) server_negotiation.cc:407] Connection header received
0808 17:02:10.759862 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.759863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.760288 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.760352 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.761605 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.762600 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.763699 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.763972 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.764033 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.765146 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.765147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.765148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.765154 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.766434 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.766735 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.766736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.766737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.766791 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.766950 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.766951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.766951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.767066 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.767166 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.767347 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.767416 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1103,"thread_start_us":56,"threads_started":1}
I20260808 17:02:10.767838 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.758054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59994 (local address 127.30.194.61:42613)
0808 17:02:10.759341 (+ 1287us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.759343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.759657 (+ 314us) server_negotiation.cc:407] Connection header received
0808 17:02:10.760627 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.760628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.760638 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.760665 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.761894 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.762536 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.763465 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.763717 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.763755 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.764968 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.764969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.764969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.764976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.766695 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.767032 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.767034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.767034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.767093 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.767288 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.767289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.767290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.767417 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.767496 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.767680 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.767754 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1274,"thread_start_us":68,"threads_started":1}
I20260808 17:02:10.770753 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.758730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55660 (local address 127.30.194.62:35661)
0808 17:02:10.760161 (+ 1431us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.760163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.764117 (+ 3954us) server_negotiation.cc:407] Connection header received
0808 17:02:10.764198 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.764199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.764209 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.764255 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.765522 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.766185 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.767112 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.767344 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.767403 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.768742 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.768743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.768743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.768750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.769919 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.770218 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.770219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.770219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.770265 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.770390 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.770390 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.770390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.770475 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.770540 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.770636 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.770693 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1418,"thread_start_us":78,"threads_started":1}
I20260808 17:02:10.772011 9622 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:02:10.773774 9622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:02:10.774456 10499 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: NON_VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: true } } }
I20260808 17:02:10.774561 10499 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:10.775832 10499 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594: attempting to promote NON_VOTER 498bf9fcca2f4ea5981c96dcad55d4c0 to VOTER
I20260808 17:02:10.775974 10499 consensus_queue.cc:237] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } }
I20260808 17:02:10.776399 10880 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 LEARNER]: Refusing update from remote peer b46b984f705b49c19aec28869ecc7594: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 17:02:10.776507 10576 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Refusing update from remote peer b46b984f705b49c19aec28869ecc7594: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 17:02:10.776721 11584 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 17:02:10.776798 10500 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 17:02:10.777558 10500 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:10.777689 10880 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:10.778340 10500 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:10.778414 10500 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:10.778465 10500 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:10.778508 10500 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:10.778540 10500 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:10.778582 10500 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:10.778657 10924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:10.778741 10924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:02:10.778898 10576 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:10.779745 10804 sys_catalog.cc:455] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b46b984f705b49c19aec28869ecc7594" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:10.779824 10804 sys_catalog.cc:458] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:02:10.783005 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.779210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45600 (local address 127.30.194.3:33505)
0808 17:02:10.779394 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.779399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.779543 (+ 144us) server_negotiation.cc:407] Connection header received
0808 17:02:10.779652 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.779655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.779670 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.779726 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:10.779915 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.780941 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.782063 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.782710 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.782761 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.782814 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.782923 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.782950 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260808 17:02:10.783012 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.778877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:33505 (local address 127.0.0.1:45600)
0808 17:02:10.779469 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.779479 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.779573 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.779801 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.779801 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.779804 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:10.779841 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.779843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.781034 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.781034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.781957 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.781960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.782778 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.782779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.782843 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.782847 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.782897 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.782942 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":463,"thread_start_us":70,"threads_started":1}
I20260808 17:02:10.783787 11593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.778949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37889 (local address 127.0.0.1:44834)
0808 17:02:10.779071 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.779079 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.779120 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.779671 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.779672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.779675 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:10.779724 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.779726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.780818 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.780819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.781926 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.781929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.783563 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.783564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.783626 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.783629 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.783688 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.783726 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260808 17:02:10.783839 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.779214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.30.194.1:37889)
0808 17:02:10.779487 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.779491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.779501 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:10.779541 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.779543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.779559 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.779802 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:10.779881 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.780725 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.782011 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.783484 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.783543 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.783593 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.783720 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.783759 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":85,"threads_started":1}
I20260808 17:02:10.783707 11449 tablet_service.cc:1467] Processing CreateTablet for tablet b5a6cf935c0f417abb4604a9e3d03d00 (DEFAULT_TABLE table=test-table [id=e939bf6512d34db490ae4f7634647bfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:10.784090 11449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a6cf935c0f417abb4604a9e3d03d00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:10.784153 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.778869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:35543 (local address 127.0.0.1:52268)
0808 17:02:10.778901 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.778912 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.778955 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.779753 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.779754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.779756 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:10.779796 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.779798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.781391 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.781392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.782460 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.782463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.783959 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.783960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.784028 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.784031 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.784060 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.784094 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 17:02:10.784503 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.779004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52268 (local address 127.30.194.2:35543)
0808 17:02:10.779564 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.779569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.779578 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:10.779616 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.779618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.779633 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.779926 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:10.779986 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.781328 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.782569 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.783799 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.784301 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.784386 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.784410 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.784439 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":79,"threads_started":1}
I20260808 17:02:10.784490 11031 tablet_service.cc:1467] Processing CreateTablet for tablet b5a6cf935c0f417abb4604a9e3d03d00 (DEFAULT_TABLE table=test-table [id=e939bf6512d34db490ae4f7634647bfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:10.784986 11031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a6cf935c0f417abb4604a9e3d03d00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:10.785066 11236 tablet_service.cc:1467] Processing CreateTablet for tablet b5a6cf935c0f417abb4604a9e3d03d00 (DEFAULT_TABLE table=test-table [id=e939bf6512d34db490ae4f7634647bfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:10.785463 11236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a6cf935c0f417abb4604a9e3d03d00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:10.787504 11601 tablet_bootstrap.cc:492] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: Bootstrap starting.
I20260808 17:02:10.788196 11601 tablet_bootstrap.cc:654] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:10.788270 11602 tablet_bootstrap.cc:492] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: Bootstrap starting.
I20260808 17:02:10.788738 11601 log.cc:824] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:10.788882 11602 tablet_bootstrap.cc:654] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:10.789184 11602 log.cc:824] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:10.789251 11604 tablet_bootstrap.cc:492] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: Bootstrap starting.
I20260808 17:02:10.789522 11601 tablet_bootstrap.cc:492] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: No bootstrap required, opened a new log
I20260808 17:02:10.789565 11601 ts_tablet_manager.cc:1397] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:10.789769 11604 tablet_bootstrap.cc:654] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:10.789949 11602 tablet_bootstrap.cc:492] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: No bootstrap required, opened a new log
I20260808 17:02:10.789979 11604 log.cc:824] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:10.790002 11602 ts_tablet_manager.cc:1397] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:10.790546 11604 tablet_bootstrap.cc:492] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: No bootstrap required, opened a new log
I20260808 17:02:10.790586 11604 ts_tablet_manager.cc:1397] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:10.791822 11604 raft_consensus.cc:348] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.791929 11604 raft_consensus.cc:374] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:10.791949 11604 raft_consensus.cc:729] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 925364577a1542148cb6c46abf5ab9ec, State: Initialized, Role: FOLLOWER
I20260808 17:02:10.791951 11601 raft_consensus.cc:348] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.792034 11604 consensus_queue.cc:260] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.792092 11601 raft_consensus.cc:374] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:10.792136 11601 raft_consensus.cc:729] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d1b745ca0b424b920ed368038dc8d2, State: Initialized, Role: FOLLOWER
I20260808 17:02:10.792295 11604 ts_tablet_manager.cc:1428] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:02:10.792254 11601 consensus_queue.cc:260] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.792409 11337 heartbeater.cc:503] Master 127.30.194.62:35661 was elected leader, sending a full tablet report...
I20260808 17:02:10.792510 11601 ts_tablet_manager.cc:1428] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:02:10.792815 11563 heartbeater.cc:503] Master 127.30.194.62:35661 was elected leader, sending a full tablet report...
I20260808 17:02:10.793712 11602 raft_consensus.cc:348] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.793866 11602 raft_consensus.cc:374] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:10.793895 11602 raft_consensus.cc:729] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4b7243cd2ab4fd39a4a4968c110d86e, State: Initialized, Role: FOLLOWER
I20260808 17:02:10.793998 11602 consensus_queue.cc:260] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.794203 11602 ts_tablet_manager.cc:1428] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 17:02:10.794251 11133 heartbeater.cc:503] Master 127.30.194.62:35661 was elected leader, sending a full tablet report...
W20260808 17:02:10.845228 11344 tablet.cc:2369] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:02:10.847887 11612 raft_consensus.cc:482] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:10.847997 11612 raft_consensus.cc:504] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.848076 11613 raft_consensus.cc:482] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:10.848169 11613 raft_consensus.cc:504] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.848420 11612 leader_election.cc:288] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d1b745ca0b424b920ed368038dc8d2 (127.30.194.3:33505), b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889)
I20260808 17:02:10.848439 11613 leader_election.cc:288] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 925364577a1542148cb6c46abf5ab9ec (127.30.194.2:35543), b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889)
I20260808 17:02:10.851953 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.848559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:33505 (local address 127.30.194.2:49095)
0808 17:02:10.848690 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.848699 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.848731 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.848943 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.848943 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.848945 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:10.849013 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.849014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.849919 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.849920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.851041 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.851043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.851724 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.851725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.851801 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.851804 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.851844 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.851883 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260808 17:02:10.852134 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.848780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:49095 (local address 127.30.194.3:33505)
0808 17:02:10.848817 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.848818 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.848826 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:10.848846 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.848847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.848855 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.848899 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:10.849101 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.849840 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.851129 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.851630 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.851688 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.851697 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.852043 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.852081 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:02:10.852275 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.848595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:35543 (local address 127.30.194.3:59903)
0808 17:02:10.848670 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.848681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.848720 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.849009 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.849010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.849012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:10.849077 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.849079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.849952 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.849953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.851070 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.851073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.852021 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.852022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.852105 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.852110 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.852164 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.852203 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 17:02:10.852350 11350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.848584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37889 (local address 127.30.194.2:37209)
0808 17:02:10.848855 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.848864 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.848894 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.849114 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.849115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.849116 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:10.849455 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.849457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.850332 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.850333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.851347 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.851349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.852129 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.852130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.852205 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.852208 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.852254 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.852286 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241}
I20260808 17:02:10.852530 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.848804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:59903 (local address 127.30.194.2:35543)
0808 17:02:10.848859 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.848860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.848869 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:10.848892 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.848893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.848902 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.848957 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:10.849180 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.849850 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.851392 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.851907 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.851978 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.851985 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.852436 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.852473 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:02:10.852566 11568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.848609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37889 (local address 127.30.194.3:37763)
0808 17:02:10.848677 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:10.848687 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:10.848725 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:10.849366 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:10.849367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:10.849369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:10.849437 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.849439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.850361 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.850362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.851362 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:10.851365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.852314 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.852314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:10.852392 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.852395 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:10.852459 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:02:10.852497 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 17:02:10.852659 11481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a6cf935c0f417abb4604a9e3d03d00" candidate_uuid: "925364577a1542148cb6c46abf5ab9ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d1b745ca0b424b920ed368038dc8d2" is_pre_election: true
I20260808 17:02:10.852754 11481 raft_consensus.cc:2454] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 925364577a1542148cb6c46abf5ab9ec in term 0.
I20260808 17:02:10.852787 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a6cf935c0f417abb4604a9e3d03d00" candidate_uuid: "99d1b745ca0b424b920ed368038dc8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "925364577a1542148cb6c46abf5ab9ec" is_pre_election: true
I20260808 17:02:10.852882 11265 raft_consensus.cc:2454] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d1b745ca0b424b920ed368038dc8d2 in term 0.
I20260808 17:02:10.852942 11616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.848932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:37763 (local address 127.30.194.1:37889)
0808 17:02:10.849222 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.849224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.849232 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:10.849253 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.849254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.849264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.849502 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:10.849537 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.850254 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.851461 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.852123 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.852313 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.852567 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.852820 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.852859 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":261,"thread_start_us":79,"threads_started":1}
I20260808 17:02:10.852933 11169 leader_election.cc:302] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 925364577a1542148cb6c46abf5ab9ec, 99d1b745ca0b424b920ed368038dc8d2; no voters:
I20260808 17:02:10.853065 11612 raft_consensus.cc:2781] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:10.853075 11377 leader_election.cc:302] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 925364577a1542148cb6c46abf5ab9ec, 99d1b745ca0b424b920ed368038dc8d2; no voters:
I20260808 17:02:10.853132 11612 raft_consensus.cc:482] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:10.853159 11612 raft_consensus.cc:3037] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:10.853199 11062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a6cf935c0f417abb4604a9e3d03d00" candidate_uuid: "99d1b745ca0b424b920ed368038dc8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" is_pre_election: true
I20260808 17:02:10.853333 11062 raft_consensus.cc:2454] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d1b745ca0b424b920ed368038dc8d2 in term 0.
I20260808 17:02:10.853576 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.848911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:37209 (local address 127.30.194.1:37889)
0808 17:02:10.848975 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.848976 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.848984 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:10.849003 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.849004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.849012 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.849054 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:10.849539 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.850197 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.851425 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.851961 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.852017 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.852025 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.853488 (+ 1463us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.853526 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":42}
I20260808 17:02:10.853662 11613 raft_consensus.cc:2781] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:10.853724 11613 raft_consensus.cc:482] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:10.853749 11613 raft_consensus.cc:3037] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:10.853735 11062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a6cf935c0f417abb4604a9e3d03d00" candidate_uuid: "925364577a1542148cb6c46abf5ab9ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" is_pre_election: true
I20260808 17:02:10.853809 11062 raft_consensus.cc:2454] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 925364577a1542148cb6c46abf5ab9ec in term 0.
I20260808 17:02:10.854040 11612 raft_consensus.cc:504] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.854162 11612 leader_election.cc:288] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [CANDIDATE]: Term 1 election: Requested vote from peers 99d1b745ca0b424b920ed368038dc8d2 (127.30.194.3:33505), b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889)
I20260808 17:02:10.854315 11062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a6cf935c0f417abb4604a9e3d03d00" candidate_uuid: "925364577a1542148cb6c46abf5ab9ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e"
I20260808 17:02:10.854332 11481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a6cf935c0f417abb4604a9e3d03d00" candidate_uuid: "925364577a1542148cb6c46abf5ab9ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d1b745ca0b424b920ed368038dc8d2"
I20260808 17:02:10.854413 11062 raft_consensus.cc:3037] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:10.854686 11613 raft_consensus.cc:504] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.854825 11481 raft_consensus.cc:2379] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 925364577a1542148cb6c46abf5ab9ec in current term 1: Already voted for candidate 99d1b745ca0b424b920ed368038dc8d2 in this term.
I20260808 17:02:10.854846 11613 leader_election.cc:288] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [CANDIDATE]: Term 1 election: Requested vote from peers 925364577a1542148cb6c46abf5ab9ec (127.30.194.2:35543), b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889)
I20260808 17:02:10.854975 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a6cf935c0f417abb4604a9e3d03d00" candidate_uuid: "99d1b745ca0b424b920ed368038dc8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "925364577a1542148cb6c46abf5ab9ec"
I20260808 17:02:10.855057 11265 raft_consensus.cc:2379] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d1b745ca0b424b920ed368038dc8d2 in current term 1: Already voted for candidate 925364577a1542148cb6c46abf5ab9ec in this term.
I20260808 17:02:10.855109 11060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a6cf935c0f417abb4604a9e3d03d00" candidate_uuid: "99d1b745ca0b424b920ed368038dc8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e"
I20260808 17:02:10.855314 11062 raft_consensus.cc:2454] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 925364577a1542148cb6c46abf5ab9ec in term 1.
I20260808 17:02:10.855420 11372 leader_election.cc:302] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d1b745ca0b424b920ed368038dc8d2; no voters: 925364577a1542148cb6c46abf5ab9ec, b4b7243cd2ab4fd39a4a4968c110d86e
I20260808 17:02:10.855468 11165 leader_election.cc:302] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 925364577a1542148cb6c46abf5ab9ec, b4b7243cd2ab4fd39a4a4968c110d86e; no voters: 99d1b745ca0b424b920ed368038dc8d2
I20260808 17:02:10.855494 11613 raft_consensus.cc:2726] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 17:02:10.855562 11612 raft_consensus.cc:2781] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:10.855706 11612 raft_consensus.cc:686] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 1 LEADER]: Becoming Leader. State: Replica: 925364577a1542148cb6c46abf5ab9ec, State: Running, Role: LEADER
I20260808 17:02:10.855796 11612 consensus_queue.cc:237] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } }
I20260808 17:02:10.856753 9622 catalog_manager.cc:5495] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 925364577a1542148cb6c46abf5ab9ec (127.30.194.2). New cstate: current_term: 1 leader_uuid: "925364577a1542148cb6c46abf5ab9ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } health_report { overall_health: UNKNOWN } } }
I20260808 17:02:10.873198 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.871279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.30.194.2:35543)
0808 17:02:10.871327 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.871328 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.871336 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:10.871388 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.871389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.871402 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.871455 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:10.871541 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.871987 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.872651 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.872824 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.872868 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.872960 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:10.873044 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:10.873124 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.873140 (+ 16us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.873155 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:02:10.874646 11481 raft_consensus.cc:1261] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 FOLLOWER]: Refusing update from remote peer 925364577a1542148cb6c46abf5ab9ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:10.874648 11062 raft_consensus.cc:1261] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [term 1 FOLLOWER]: Refusing update from remote peer 925364577a1542148cb6c46abf5ab9ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:10.874879 11617 consensus_queue.cc:1035] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:10.874982 11612 consensus_queue.cc:1035] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:10.877955 11620 mvcc.cc:205] Tried to move back new op lower bound from 7316310142460563456 to 7316310142385897472. Current Snapshot: MvccSnapshot[applied={T|T < 7316310142460563456}]
I20260808 17:02:10.879396 11626 mvcc.cc:205] Tried to move back new op lower bound from 7316310142460563456 to 7316310142385897472. Current Snapshot: MvccSnapshot[applied={T|T < 7316310142460563456}]
I20260808 17:02:10.879663 11622 mvcc.cc:205] Tried to move back new op lower bound from 7316310142460563456 to 7316310142385897472. Current Snapshot: MvccSnapshot[applied={T|T < 7316310142460563456 or (T in {7316310142460563456})}]
I20260808 17:02:10.880671 9622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:10.880745 9622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:10.882129 9622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b46b984f705b49c19aec28869ecc7594: Sending DeleteTablet for 3 replicas of tablet b5a6cf935c0f417abb4604a9e3d03d00
I20260808 17:02:10.882413 11449 tablet_service.cc:1514] Processing DeleteTablet for tablet b5a6cf935c0f417abb4604a9e3d03d00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:10 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:45600
I20260808 17:02:10.882467 11031 tablet_service.cc:1514] Processing DeleteTablet for tablet b5a6cf935c0f417abb4604a9e3d03d00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:10 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:44834
I20260808 17:02:10.882650 11634 tablet_replica.cc:330] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: stopping tablet replica
I20260808 17:02:10.882666 11635 tablet_replica.cc:330] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: stopping tablet replica
I20260808 17:02:10.882705 11236 tablet_service.cc:1514] Processing DeleteTablet for tablet b5a6cf935c0f417abb4604a9e3d03d00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:10 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:52268
I20260808 17:02:10.882752 11635 raft_consensus.cc:2229] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:10.882844 11635 raft_consensus.cc:2258] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:10.882839 11634 raft_consensus.cc:2229] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:10.882979 11634 raft_consensus.cc:2258] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:10.883296 11635 ts_tablet_manager.cc:1905] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:10.883450 11634 ts_tablet_manager.cc:1905] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:10.884065 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 9568
I20260808 17:02:10.887188 11635 ts_tablet_manager.cc:1918] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:10.887259 11635 log.cc:1196] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/b5a6cf935c0f417abb4604a9e3d03d00
I20260808 17:02:10.887454 11635 ts_tablet_manager.cc:1939] T b5a6cf935c0f417abb4604a9e3d03d00 P b4b7243cd2ab4fd39a4a4968c110d86e: Deleting consensus metadata
I20260808 17:02:10.887595 11636 tablet_replica.cc:330] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: stopping tablet replica
I20260808 17:02:10.887665 11636 raft_consensus.cc:2229] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:02:10.887727 11636 raft_consensus.cc:2258] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:10.890285 11636 ts_tablet_manager.cc:1905] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:10.891580 11636 ts_tablet_manager.cc:1918] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:10.891637 11636 log.cc:1196] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/b5a6cf935c0f417abb4604a9e3d03d00
I20260808 17:02:10.891858 11636 ts_tablet_manager.cc:1939] T b5a6cf935c0f417abb4604a9e3d03d00 P 925364577a1542148cb6c46abf5ab9ec: Deleting consensus metadata
I20260808 17:02:10.894142 11634 ts_tablet_manager.cc:1918] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:10.894265 11634 log.cc:1196] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/b5a6cf935c0f417abb4604a9e3d03d00
I20260808 17:02:10.894536 11634 ts_tablet_manager.cc:1939] T b5a6cf935c0f417abb4604a9e3d03d00 P 99d1b745ca0b424b920ed368038dc8d2: Deleting consensus metadata
W20260808 17:02:10.899963 11377 connection.cc:535] server connection from 127.0.0.1:45600 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 17:02:10.900410 11166 connection.cc:535] server connection from 127.0.0.1:52268 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 17:02:10.900580 10963 connection.cc:441] server connection from 127.0.0.1:44834 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44834 (request call id 1) could send its response
I20260808 17:02:10.902580 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.883529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48034 (local address 127.30.194.60:34603)
0808 17:02:10.883568 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.883570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.895024 (+ 11454us) server_negotiation.cc:407] Connection header received
0808 17:02:10.895061 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.895062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.895077 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.895142 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.895780 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.896537 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.897522 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.897850 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.897914 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.899325 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.899327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.899327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.899334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.901242 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.901615 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.901617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.901617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.901678 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.901902 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.901903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.901904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.902025 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.902098 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.902399 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.902484 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:02:10.909233 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.902294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48046 (local address 127.30.194.60:34603)
0808 17:02:10.902335 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.902336 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.902405 (+ 69us) server_negotiation.cc:407] Connection header received
0808 17:02:10.902487 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.902488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.902498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.902524 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.903128 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.903834 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.904920 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.905247 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.905315 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.906664 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.906665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.906665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.906672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.908150 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.908520 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.908520 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.908521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.908574 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.908750 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.908751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.908751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.908864 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.908956 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.909055 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.909145 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:02:10.909334 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:10.902002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.30.194.61:42613)
0808 17:02:10.902047 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:10.902050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:10.902198 (+ 148us) server_negotiation.cc:407] Connection header received
0808 17:02:10.902305 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:10.902307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:10.902318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:10.902347 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:10.903237 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.903942 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.905150 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:10.905365 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:10.905443 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:10.906715 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:10.906716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:10.906717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:10.906723 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:10.908110 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.908545 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.908546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.908546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.908601 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:10.908788 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:10.908789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:10.908789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:10.908894 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:10.908977 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:10.909175 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 17:02:10.909246 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:02:11.879194 11650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:11.878934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.3:52297)
0808 17:02:11.879058 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:11.879143 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260808 17:02:11.879446 11651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:11.879219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.1:38633)
0808 17:02:11.879330 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:11.879402 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":50,"threads_started":1}
W20260808 17:02:11.879467 11563 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:35661 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
W20260808 17:02:11.879612 11133 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:35661 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
I20260808 17:02:11.880281 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:11.880109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.2:60897)
0808 17:02:11.880193 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:11.880248 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":42,"threads_started":1}
W20260808 17:02:11.880432 11337 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:35661 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
I20260808 17:02:12.382354 11654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b46b984f705b49c19aec28869ecc7594)
I20260808 17:02:12.382473 11654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } }
I20260808 17:02:12.382706 11654 leader_election.cc:288] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661), c163d97601c94054ab26aacaf129394a (127.30.194.61:42613)
I20260808 17:02:12.382820 11655 raft_consensus.cc:482] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b46b984f705b49c19aec28869ecc7594)
I20260808 17:02:12.382966 11655 raft_consensus.cc:504] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } }
I20260808 17:02:12.383041 11658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.382830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56084)
0808 17:02:12.382946 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.382998 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":48,"threads_started":1}
W20260808 17:02:12.383183 10844 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
I20260808 17:02:12.383553 11655 leader_election.cc:288] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661), 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60:34603)
I20260808 17:02:12.383579 11658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.383485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56104)
0808 17:02:12.383529 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.383556 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 17:02:12.383652 10844 leader_election.cc:334] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661): Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
I20260808 17:02:12.383821 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.383448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56098)
0808 17:02:12.383730 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.383788 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":38,"threads_started":1}
W20260808 17:02:12.383924 10538 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
I20260808 17:02:12.384976 11665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.384844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56120)
0808 17:02:12.384900 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.384945 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 17:02:12.385120 10538 leader_election.cc:334] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661): Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
I20260808 17:02:12.386296 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.382842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42613 (local address 127.0.0.1:60012)
0808 17:02:12.382986 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.382996 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:12.383051 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:12.383599 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:12.383600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:12.383603 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:12.383767 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:12.383770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:12.384600 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:12.384601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:12.385541 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:12.385544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:12.386069 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:12.386070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:12.386135 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:12.386139 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:02:12.386183 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 17:02:12.386227 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":46,"threads_started":1}
I20260808 17:02:12.386305 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.382895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60012 (local address 127.30.194.61:42613)
0808 17:02:12.383023 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:12.383026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:12.383039 (+ 13us) server_negotiation.cc:407] Connection header received
0808 17:02:12.383352 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:12.383353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:12.383366 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:12.383548 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:12.383857 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:12.384525 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:12.385617 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:12.386000 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:12.386062 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:12.386073 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:12.386193 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 17:02:12.386240 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":42,"threads_started":1}
I20260808 17:02:12.386412 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.383461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:34603 (local address 127.0.0.1:48050)
0808 17:02:12.383880 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.383889 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:12.383947 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:12.384098 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:12.384099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:12.384101 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:12.384161 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:12.384162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:12.384777 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:12.384778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:12.385670 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:12.385673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:12.386187 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:12.386188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:12.386260 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:12.386265 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:02:12.386308 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 17:02:12.386347 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":65,"threads_started":1}
I20260808 17:02:12.386415 11666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.383634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.30.194.60:34603)
0808 17:02:12.383797 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:12.383799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:12.383993 (+ 194us) server_negotiation.cc:407] Connection header received
0808 17:02:12.384014 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:12.384015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:12.384025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:12.384090 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:12.384232 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:12.384701 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:12.385751 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:12.386119 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:12.386177 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:12.386186 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:12.386318 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 17:02:12.386355 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260808 17:02:12.386997 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c163d97601c94054ab26aacaf129394a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" is_pre_election: true
I20260808 17:02:12.387058 10576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c163d97601c94054ab26aacaf129394a" is_pre_election: true
I20260808 17:02:12.387094 10880 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c163d97601c94054ab26aacaf129394a in term 1.
I20260808 17:02:12.387142 10576 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498bf9fcca2f4ea5981c96dcad55d4c0 in term 1.
I20260808 17:02:12.387279 10537 leader_election.cc:302] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 498bf9fcca2f4ea5981c96dcad55d4c0, c163d97601c94054ab26aacaf129394a; no voters: b46b984f705b49c19aec28869ecc7594
I20260808 17:02:12.387279 10843 leader_election.cc:302] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 498bf9fcca2f4ea5981c96dcad55d4c0, c163d97601c94054ab26aacaf129394a; no voters: b46b984f705b49c19aec28869ecc7594
I20260808 17:02:12.387389 11654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 17:02:12.387434 11654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b46b984f705b49c19aec28869ecc7594)
I20260808 17:02:12.387461 11654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:02:12.388131 11654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } }
I20260808 17:02:12.388247 11654 leader_election.cc:288] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [CANDIDATE]: Term 2 election: Requested vote from peers b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661), c163d97601c94054ab26aacaf129394a (127.30.194.61:42613)
I20260808 17:02:12.388399 11655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 17:02:12.388441 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.388345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56124)
0808 17:02:12.388395 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.388418 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 17:02:12.388393 10576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c163d97601c94054ab26aacaf129394a"
I20260808 17:02:12.388474 11655 raft_consensus.cc:482] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Starting leader election (detected failure of leader b46b984f705b49c19aec28869ecc7594)
I20260808 17:02:12.388496 11655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:02:12.388697 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.388615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56134)
0808 17:02:12.388654 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.388676 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 17:02:12.388760 10844 leader_election.cc:334] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661): Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
I20260808 17:02:12.389230 11655 raft_consensus.cc:504] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } }
I20260808 17:02:12.389341 10576 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 498bf9fcca2f4ea5981c96dcad55d4c0 in current term 2: Already voted for candidate c163d97601c94054ab26aacaf129394a in this term.
I20260808 17:02:12.389350 11655 leader_election.cc:288] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 2 election: Requested vote from peers b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661), 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60:34603)
I20260808 17:02:12.389519 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.389423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56148)
0808 17:02:12.389466 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.389498 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 17:02:12.389559 10843 leader_election.cc:302] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 498bf9fcca2f4ea5981c96dcad55d4c0; no voters: b46b984f705b49c19aec28869ecc7594, c163d97601c94054ab26aacaf129394a
I20260808 17:02:12.389690 11654 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 17:02:12.389701 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c163d97601c94054ab26aacaf129394a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0"
I20260808 17:02:12.389828 10880 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c163d97601c94054ab26aacaf129394a in current term 2: Already voted for candidate 498bf9fcca2f4ea5981c96dcad55d4c0 in this term.
I20260808 17:02:12.389858 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.389736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56152)
0808 17:02:12.389806 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.389836 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 17:02:12.389935 10538 leader_election.cc:334] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661): Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
I20260808 17:02:12.389993 10537 leader_election.cc:302] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c163d97601c94054ab26aacaf129394a; no voters: 498bf9fcca2f4ea5981c96dcad55d4c0, b46b984f705b49c19aec28869ecc7594
I20260808 17:02:12.390085 11655 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 17:02:12.880018 11668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.879772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.3:56507)
0808 17:02:12.879931 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.879967 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260808 17:02:12.880316 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.879929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.1:54007)
0808 17:02:12.880231 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.880270 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":53,"threads_started":1}
I20260808 17:02:12.880858 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:12.880684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.2:54585)
0808 17:02:12.880794 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:12.880824 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260808 17:02:13.880733 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:13.880514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.3:43115)
0808 17:02:13.880630 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:13.880676 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":79,"threads_started":1}
I20260808 17:02:13.881111 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:13.880729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.1:38841)
0808 17:02:13.881004 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:13.881043 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":54,"threads_started":1}
I20260808 17:02:13.881445 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:13.881230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.2:36483)
0808 17:02:13.881369 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:13.881405 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":50,"threads_started":1}
I20260808 17:02:14.048190 11677 raft_consensus.cc:482] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:14.048285 11677 raft_consensus.cc:504] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } }
I20260808 17:02:14.048436 11677 leader_election.cc:288] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661), 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60:34603)
I20260808 17:02:14.048626 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c163d97601c94054ab26aacaf129394a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" is_pre_election: true
I20260808 17:02:14.048677 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.048521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56364)
0808 17:02:14.048614 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.048644 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":48,"threads_started":1}
I20260808 17:02:14.048709 10880 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c163d97601c94054ab26aacaf129394a in term 2.
I20260808 17:02:14.048831 10537 leader_election.cc:302] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498bf9fcca2f4ea5981c96dcad55d4c0, c163d97601c94054ab26aacaf129394a; no voters:
I20260808 17:02:14.048928 11677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 17:02:14.048976 11677 raft_consensus.cc:482] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:14.049006 11677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 2 FOLLOWER]: Advancing to term 3
I20260808 17:02:14.049121 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.048986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56378)
0808 17:02:14.049044 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.049074 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 17:02:14.049185 10538 leader_election.cc:334] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661): Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
I20260808 17:02:14.049772 11677 raft_consensus.cc:504] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } }
I20260808 17:02:14.049877 11677 leader_election.cc:288] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 3 election: Requested vote from peers b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661), 498bf9fcca2f4ea5981c96dcad55d4c0 (127.30.194.60:34603)
I20260808 17:02:14.050040 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.049963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56382)
0808 17:02:14.049994 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.050018 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 17:02:14.050014 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c163d97601c94054ab26aacaf129394a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0"
I20260808 17:02:14.050078 10880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 2 FOLLOWER]: Advancing to term 3
I20260808 17:02:14.050344 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.050260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56384)
0808 17:02:14.050301 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.050329 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 17:02:14.050407 10538 leader_election.cc:334] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661): Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
I20260808 17:02:14.050797 10880 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c163d97601c94054ab26aacaf129394a in term 3.
I20260808 17:02:14.050977 10537 leader_election.cc:302] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 498bf9fcca2f4ea5981c96dcad55d4c0, c163d97601c94054ab26aacaf129394a; no voters: b46b984f705b49c19aec28869ecc7594
I20260808 17:02:14.051059 11677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 3 FOLLOWER]: Leader election won for term 3
I20260808 17:02:14.051163 11677 raft_consensus.cc:686] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [term 3 LEADER]: Becoming Leader. State: Replica: c163d97601c94054ab26aacaf129394a, State: Running, Role: LEADER
I20260808 17:02:14.051292 11677 consensus_queue.cc:237] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } }
I20260808 17:02:14.051631 11683 sys_catalog.cc:455] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c163d97601c94054ab26aacaf129394a. Latest consensus state: current_term: 3 leader_uuid: "c163d97601c94054ab26aacaf129394a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:14.051738 11683 sys_catalog.cc:458] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:14.051931 11688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:14.052317 11688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:14.052384 11688 catalog_manager.cc:1259] Loaded cluster ID: d4aac8015e2d4470b35e4009d8309005
I20260808 17:02:14.052414 11688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:14.052589 11688 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:14.052660 11688 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Loaded TSK: 0
I20260808 17:02:14.052805 11688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:02:14.133347 10557 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:02:14.135006 10557 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:02:14.135982 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.135883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56404)
0808 17:02:14.135927 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.135954 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 17:02:14.136003 10880 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [term 3 FOLLOWER]: Refusing update from remote peer c163d97601c94054ab26aacaf129394a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 17:02:14.136303 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.136213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.0.0.1:56416)
0808 17:02:14.136259 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.136284 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 17:02:14.136252 11677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [LEADER]: Connected to new peer: Peer: permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 17:02:14.136387 10538 consensus_peers.cc:487] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a -> Peer b46b984f705b49c19aec28869ecc7594 (127.30.194.62:35661): Couldn't send request to peer b46b984f705b49c19aec28869ecc7594. Status: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 17:02:14.136845 11690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c163d97601c94054ab26aacaf129394a. Latest consensus state: current_term: 3 leader_uuid: "c163d97601c94054ab26aacaf129394a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:14.136936 11690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:02:14.137280 11689 mvcc.cc:205] Tried to move back new op lower bound from 7316310155819343872 to 7316310155474608128. Current Snapshot: MvccSnapshot[applied={T|T < 7316310155819343872}]
I20260808 17:02:14.137368 11683 sys_catalog.cc:455] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c163d97601c94054ab26aacaf129394a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:14.137461 11683 sys_catalog.cc:458] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:14.137596 11690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c163d97601c94054ab26aacaf129394a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b46b984f705b49c19aec28869ecc7594" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35661 } } peers { permanent_uuid: "c163d97601c94054ab26aacaf129394a" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42613 } attrs { promote: false } } peers { permanent_uuid: "498bf9fcca2f4ea5981c96dcad55d4c0" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 34603 } attrs { promote: false } } }
I20260808 17:02:14.137673 11690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 498bf9fcca2f4ea5981c96dcad55d4c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:02:14.142706 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.139120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37889 (local address 127.0.0.1:44844)
0808 17:02:14.139302 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.139312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:14.139367 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:14.139638 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:14.139639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:14.139641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:14.139690 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:14.139692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.140564 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.140564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:14.141725 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:14.141727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.142438 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.142439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:14.142499 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:14.142503 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:02:14.142549 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 17:02:14.142600 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":56}
I20260808 17:02:14.142791 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.139178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:35543 (local address 127.0.0.1:52288)
0808 17:02:14.139303 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.139312 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:14.139367 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:14.139683 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:14.139684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:14.139686 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:14.139731 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:14.139732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.140495 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.140496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:14.141949 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:14.141951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.142529 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.142530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:14.142627 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:14.142631 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:02:14.142681 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:02:14.142719 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":50,"thread_start_us":91,"threads_started":1}
I20260808 17:02:14.142804 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.139387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.30.194.2:35543)
0808 17:02:14.139488 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:14.139490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:14.139497 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:14.139523 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:14.139525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:14.139535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:14.139597 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:14.139807 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.140430 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.142015 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.142456 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.142523 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:14.142534 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:14.142694 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 17:02:14.142738 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":48,"threads_started":1}
I20260808 17:02:14.142967 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.139202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.30.194.1:37889)
0808 17:02:14.139497 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:14.139499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:14.139507 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:14.139548 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:14.139549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:14.139560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:14.139635 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:14.139774 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.140503 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.141797 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.142334 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.142394 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:14.142406 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:14.142679 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 17:02:14.142731 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":70,"threads_started":1}
I20260808 17:02:14.143049 11704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.139360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45608 (local address 127.30.194.3:33505)
0808 17:02:14.139711 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:14.139713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:14.139720 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:14.139744 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:14.139745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:14.139756 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:14.139828 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:14.139975 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.140677 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.141696 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.142284 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.142394 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:14.142406 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:14.142938 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 17:02:14.142987 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":52,"threads_started":1}
I20260808 17:02:14.142652 11701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.139180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:33505 (local address 127.0.0.1:45608)
0808 17:02:14.139438 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.139447 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:14.139481 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:14.139871 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:14.139872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:14.139875 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:14.139906 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:14.139908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.140749 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.140750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:14.141628 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:14.141631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.142396 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.142397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:14.142459 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:14.142465 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:02:14.142516 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:02:14.142556 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":87,"threads_started":1}
I20260808 17:02:14.143224 11031 tablet_service.cc:1467] Processing CreateTablet for tablet ac385d5f0b0d47d494372988e0b07377 (DEFAULT_TABLE table=test-table [id=091ec6829b5141c0908386d49d6b04ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:14.143281 11236 tablet_service.cc:1467] Processing CreateTablet for tablet ac385d5f0b0d47d494372988e0b07377 (DEFAULT_TABLE table=test-table [id=091ec6829b5141c0908386d49d6b04ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:14.143397 11031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac385d5f0b0d47d494372988e0b07377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:14.143412 11236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac385d5f0b0d47d494372988e0b07377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:14.143502 11449 tablet_service.cc:1467] Processing CreateTablet for tablet ac385d5f0b0d47d494372988e0b07377 (DEFAULT_TABLE table=test-table [id=091ec6829b5141c0908386d49d6b04ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:14.143615 11449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac385d5f0b0d47d494372988e0b07377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:14.144842 11708 tablet_bootstrap.cc:492] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e: Bootstrap starting.
I20260808 17:02:14.145219 11709 tablet_bootstrap.cc:492] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2: Bootstrap starting.
I20260808 17:02:14.145289 11708 tablet_bootstrap.cc:654] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:14.145569 11709 tablet_bootstrap.cc:654] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:14.145812 11708 tablet_bootstrap.cc:492] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e: No bootstrap required, opened a new log
I20260808 17:02:14.145856 11708 ts_tablet_manager.cc:1397] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:14.145884 11707 tablet_bootstrap.cc:492] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec: Bootstrap starting.
I20260808 17:02:14.146005 11708 raft_consensus.cc:348] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } } peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } }
I20260808 17:02:14.146066 11708 raft_consensus.cc:374] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:14.146085 11708 raft_consensus.cc:729] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4b7243cd2ab4fd39a4a4968c110d86e, State: Initialized, Role: FOLLOWER
I20260808 17:02:14.146135 11709 tablet_bootstrap.cc:492] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2: No bootstrap required, opened a new log
I20260808 17:02:14.146130 11708 consensus_queue.cc:260] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } } peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } }
I20260808 17:02:14.146176 11709 ts_tablet_manager.cc:1397] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:14.146283 11707 tablet_bootstrap.cc:654] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:14.146311 11708 ts_tablet_manager.cc:1428] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:14.146327 11709 raft_consensus.cc:348] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } } peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } }
I20260808 17:02:14.146414 11709 raft_consensus.cc:374] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:14.146435 11709 raft_consensus.cc:729] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d1b745ca0b424b920ed368038dc8d2, State: Initialized, Role: FOLLOWER
I20260808 17:02:14.146493 11709 consensus_queue.cc:260] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } } peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } }
I20260808 17:02:14.146675 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.146529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.1:42457)
0808 17:02:14.146622 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.146649 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 17:02:14.147104 11707 tablet_bootstrap.cc:492] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec: No bootstrap required, opened a new log
I20260808 17:02:14.147156 11707 ts_tablet_manager.cc:1397] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:14.147156 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.147063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.3:52879)
0808 17:02:14.147103 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.147128 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 17:02:14.147179 11709 ts_tablet_manager.cc:1428] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:14.147275 11707 raft_consensus.cc:348] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } } peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } }
I20260808 17:02:14.147346 11707 raft_consensus.cc:374] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:14.147353 11564 heartbeater.cc:503] Master 127.30.194.61:42613 was elected leader, sending a full tablet report...
I20260808 17:02:14.147405 11707 raft_consensus.cc:729] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 925364577a1542148cb6c46abf5ab9ec, State: Initialized, Role: FOLLOWER
I20260808 17:02:14.147468 11707 consensus_queue.cc:260] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } } peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } }
I20260808 17:02:14.147639 11707 ts_tablet_manager.cc:1428] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:14.147979 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.147890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.2:47153)
0808 17:02:14.147929 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.147952 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 17:02:14.148126 11340 heartbeater.cc:503] Master 127.30.194.61:42613 was elected leader, sending a full tablet report...
I20260808 17:02:14.148298 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.148202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.1:39437)
0808 17:02:14.148250 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.148273 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 17:02:14.148638 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.148552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.3:32777)
0808 17:02:14.148587 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.148612 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 17:02:14.148696 11134 heartbeater.cc:503] Master 127.30.194.61:42613 was elected leader, sending a full tablet report...
I20260808 17:02:14.172716 11718 raft_consensus.cc:482] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:14.172784 11718 raft_consensus.cc:504] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } } peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } }
I20260808 17:02:14.172890 11718 leader_election.cc:288] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889), 925364577a1542148cb6c46abf5ab9ec (127.30.194.2:35543)
I20260808 17:02:14.173102 11062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac385d5f0b0d47d494372988e0b07377" candidate_uuid: "99d1b745ca0b424b920ed368038dc8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" is_pre_election: true
I20260808 17:02:14.173101 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac385d5f0b0d47d494372988e0b07377" candidate_uuid: "99d1b745ca0b424b920ed368038dc8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "925364577a1542148cb6c46abf5ab9ec" is_pre_election: true
I20260808 17:02:14.173175 11062 raft_consensus.cc:2454] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d1b745ca0b424b920ed368038dc8d2 in term 0.
I20260808 17:02:14.173188 11265 raft_consensus.cc:2454] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d1b745ca0b424b920ed368038dc8d2 in term 0.
I20260808 17:02:14.173309 11372 leader_election.cc:302] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99d1b745ca0b424b920ed368038dc8d2, b4b7243cd2ab4fd39a4a4968c110d86e; no voters:
I20260808 17:02:14.173415 11718 raft_consensus.cc:2781] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:14.173453 11718 raft_consensus.cc:482] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:14.173475 11718 raft_consensus.cc:3037] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:14.174065 11718 raft_consensus.cc:504] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } } peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } }
I20260808 17:02:14.174181 11718 leader_election.cc:288] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [CANDIDATE]: Term 1 election: Requested vote from peers b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889), 925364577a1542148cb6c46abf5ab9ec (127.30.194.2:35543)
I20260808 17:02:14.174342 11062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac385d5f0b0d47d494372988e0b07377" candidate_uuid: "99d1b745ca0b424b920ed368038dc8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e"
I20260808 17:02:14.174346 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac385d5f0b0d47d494372988e0b07377" candidate_uuid: "99d1b745ca0b424b920ed368038dc8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "925364577a1542148cb6c46abf5ab9ec"
I20260808 17:02:14.174402 11265 raft_consensus.cc:3037] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:14.174403 11062 raft_consensus.cc:3037] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:14.174949 11062 raft_consensus.cc:2454] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d1b745ca0b424b920ed368038dc8d2 in term 1.
I20260808 17:02:14.175089 11372 leader_election.cc:302] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99d1b745ca0b424b920ed368038dc8d2, b4b7243cd2ab4fd39a4a4968c110d86e; no voters:
I20260808 17:02:14.175117 11265 raft_consensus.cc:2454] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d1b745ca0b424b920ed368038dc8d2 in term 1.
I20260808 17:02:14.175173 11718 raft_consensus.cc:2781] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:14.175369 11718 raft_consensus.cc:686] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 LEADER]: Becoming Leader. State: Replica: 99d1b745ca0b424b920ed368038dc8d2, State: Running, Role: LEADER
I20260808 17:02:14.175442 11718 consensus_queue.cc:237] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } } peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } }
I20260808 17:02:14.175618 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.175527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.3:58163)
0808 17:02:14.175564 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.175596 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 17:02:14.176128 10557 catalog_manager.cc:5495] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d1b745ca0b424b920ed368038dc8d2 (127.30.194.3). New cstate: current_term: 1 leader_uuid: "99d1b745ca0b424b920ed368038dc8d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99d1b745ca0b424b920ed368038dc8d2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 33505 } health_report { overall_health: HEALTHY } } }
I20260808 17:02:14.188349 11704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.186336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45620 (local address 127.30.194.3:33505)
0808 17:02:14.186380 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:14.186382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:14.186388 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:02:14.186414 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:14.186415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:14.186429 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:14.186486 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:14.186594 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.187038 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.187745 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:14.187926 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:14.187971 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:14.188072 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:14.188164 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:14.188210 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:14.188280 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 17:02:14.188295 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:02:14.189293 11062 raft_consensus.cc:1261] T ac385d5f0b0d47d494372988e0b07377 P b4b7243cd2ab4fd39a4a4968c110d86e [term 1 FOLLOWER]: Refusing update from remote peer 99d1b745ca0b424b920ed368038dc8d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:14.189347 11265 raft_consensus.cc:1261] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec [term 1 FOLLOWER]: Refusing update from remote peer 99d1b745ca0b424b920ed368038dc8d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:14.189486 11718 consensus_queue.cc:1035] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4b7243cd2ab4fd39a4a4968c110d86e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:14.189610 11720 consensus_queue.cc:1035] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "925364577a1542148cb6c46abf5ab9ec" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:14.190516 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.190314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.1:47315)
0808 17:02:14.190461 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.190488 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 17:02:14.190656 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.190417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.2:56217)
0808 17:02:14.190602 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.190629 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260808 17:02:14.191680 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.191470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.3:57489)
0808 17:02:14.191627 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.191654 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260808 17:02:14.191807 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.191555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.1:56435)
0808 17:02:14.191756 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.191783 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260808 17:02:14.192029 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.191935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35661 (local address 127.30.194.2:59971)
0808 17:02:14.191971 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.192003 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:35661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 17:02:14.194072 10557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:14.194160 10557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:14.195253 10557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c163d97601c94054ab26aacaf129394a: Sending DeleteTablet for 3 replicas of tablet ac385d5f0b0d47d494372988e0b07377
I20260808 17:02:14.195513 11236 tablet_service.cc:1514] Processing DeleteTablet for tablet ac385d5f0b0d47d494372988e0b07377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:14 UTC) from {username='oryx', principal='oryx/127.30.194.61@KRBTEST.COM'} at 127.0.0.1:52288
I20260808 17:02:14.195645 11031 tablet_service.cc:1514] Processing DeleteTablet for tablet ac385d5f0b0d47d494372988e0b07377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:14 UTC) from {username='oryx', principal='oryx/127.30.194.61@KRBTEST.COM'} at 127.0.0.1:44844
I20260808 17:02:14.195686 11757 tablet_replica.cc:330] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec: stopping tablet replica
I20260808 17:02:14.195746 11757 raft_consensus.cc:2229] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:14.195792 11757 raft_consensus.cc:2258] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:14.195840 11449 tablet_service.cc:1514] Processing DeleteTablet for tablet ac385d5f0b0d47d494372988e0b07377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:14 UTC) from {username='oryx', principal='oryx/127.30.194.61@KRBTEST.COM'} at 127.0.0.1:45608
I20260808 17:02:14.196018 11759 tablet_replica.cc:330] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2: stopping tablet replica
I20260808 17:02:14.196062 11757 ts_tablet_manager.cc:1905] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:14.196142 11759 raft_consensus.cc:2229] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:02:14.196260 11759 raft_consensus.cc:2258] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:14.196481 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 10801
I20260808 17:02:14.199488 11757 ts_tablet_manager.cc:1918] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:14.199539 11757 log.cc:1196] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/ac385d5f0b0d47d494372988e0b07377
I20260808 17:02:14.199734 11757 ts_tablet_manager.cc:1939] T ac385d5f0b0d47d494372988e0b07377 P 925364577a1542148cb6c46abf5ab9ec: Deleting consensus metadata
I20260808 17:02:14.204660 11759 ts_tablet_manager.cc:1905] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:14.206552 11759 ts_tablet_manager.cc:1918] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:14.206624 11759 log.cc:1196] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/ac385d5f0b0d47d494372988e0b07377
I20260808 17:02:14.206827 11759 ts_tablet_manager.cc:1939] T ac385d5f0b0d47d494372988e0b07377 P 99d1b745ca0b424b920ed368038dc8d2: Deleting consensus metadata
I20260808 17:02:14.207096 10538 catalog_manager.cc:4841] TS 99d1b745ca0b424b920ed368038dc8d2 (127.30.194.3:33505): tablet ac385d5f0b0d47d494372988e0b07377 (table test-table [id=091ec6829b5141c0908386d49d6b04ca]) successfully deleted
I20260808 17:02:14.209259 10537 catalog_manager.cc:4841] TS 925364577a1542148cb6c46abf5ab9ec (127.30.194.2:35543): tablet ac385d5f0b0d47d494372988e0b07377 (table test-table [id=091ec6829b5141c0908386d49d6b04ca]) successfully deleted
I20260808 17:02:14.215637 11701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:14.215521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37889 (local address 127.0.0.1:44846)
0808 17:02:14.215567 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:14.215596 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:37889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 17:02:14.215648 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 11142
W20260808 17:02:14.222345 10535 catalog_manager.cc:4584] TS b4b7243cd2ab4fd39a4a4968c110d86e (127.30.194.1:37889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ac385d5f0b0d47d494372988e0b07377: Network error: Client connection negotiation failed: client connection to 127.30.194.1:37889: connect: Connection refused (error 111)
I20260808 17:02:14.227361 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 11351
I20260808 17:02:14.236631 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 10348
I20260808 17:02:14.247701 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 10652
2026-08-08T17:02:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6326 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:02:14 dist-test-slave-2j7r krb5kdc[11764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:02:14 dist-test-slave-2j7r krb5kdc[11764](info): set up 2 sockets
Aug 08 17:02:14 dist-test-slave-2j7r krb5kdc[11764](info): commencing operation
krb5kdc: starting...
W20260808 17:02:15.410017 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:02:15 dist-test-slave-2j7r krb5kdc[11764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:02:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.62.keytab.
Entry for principal oryx/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.62.keytab.
I20260808 17:02:15.517549 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40257
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.62.keytab
--principal=oryx/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:40257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:15.597227 11787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:15.597424 11787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:15.597455 11787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:15.598578 11787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:15.598623 11787 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 17:02:15.598642 11787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:15.598661 11787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:15.598690 11787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:15.598717 11787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:15.600186 11787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:40257
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.62.keytab
--principal=oryx/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:15.600427 11787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:15.600630 11787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:15.605145 11794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:15.605166 11804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:15.605391 11787 server_base.cc:1034] running on GCE node
W20260808 17:02:15.605149 11798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:15.605654 11787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:15.605945 11787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:15.607082 11787 hybrid_clock.cc:648] HybridClock initialized: now 1786208535607065 us; error 30 us; skew 500 ppm
Aug 08 17:02:15 dist-test-slave-2j7r krb5kdc[11764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208535, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:15.609476 11787 init.cc:377] Logged in from keytab as oryx/127.30.194.62@KRBTEST.COM (short username oryx)
I20260808 17:02:15.609890 11787 webserver.cc:460] Webserver started at http://127.30.194.62:40283/ using document root <none> and password file <none>
I20260808 17:02:15.610074 11787 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:15.610119 11787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:15.610224 11787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:15.611171 11787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "22ec66a752d944968bb5086e7940db27"
format_stamp: "Formatted at 2026-08-08 17:02:15 on dist-test-slave-2j7r"
server_key: "d88c81cacfe2fcd82b758643b2810bea"
server_key_iv: "bf8bc85c23f9bf8a34433210ae5fc1dd"
server_key_version: "encryptionkey@0"
I20260808 17:02:15.611470 11787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "22ec66a752d944968bb5086e7940db27"
format_stamp: "Formatted at 2026-08-08 17:02:15 on dist-test-slave-2j7r"
server_key: "d88c81cacfe2fcd82b758643b2810bea"
server_key_iv: "bf8bc85c23f9bf8a34433210ae5fc1dd"
server_key_version: "encryptionkey@0"
I20260808 17:02:15.612715 11787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:02:15.613377 11812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:15.613528 11787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:15.613617 11787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "22ec66a752d944968bb5086e7940db27"
format_stamp: "Formatted at 2026-08-08 17:02:15 on dist-test-slave-2j7r"
server_key: "d88c81cacfe2fcd82b758643b2810bea"
server_key_iv: "bf8bc85c23f9bf8a34433210ae5fc1dd"
server_key_version: "encryptionkey@0"
I20260808 17:02:15.613680 11787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:15.623164 11787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:15.625231 11787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:15.625370 11787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:15.629988 11787 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:40257
I20260808 17:02:15.630057 11910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:40257 every 8 connection(s)
I20260808 17:02:15.630337 11787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:15.630925 11911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:15.631333 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 11787
I20260808 17:02:15.631445 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:15.631632 31496 external_mini_cluster.cc:1428] Setting key f2a6abe0e5c8d6f2015fac6998ab21c0
I20260808 17:02:15.633663 11911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27: Bootstrap starting.
I20260808 17:02:15.634235 11911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:15.634488 11911 log.cc:824] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:15.635106 11911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27: No bootstrap required, opened a new log
I20260808 17:02:15.636812 11911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ec66a752d944968bb5086e7940db27" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40257 } }
I20260808 17:02:15.636946 11911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:15.636981 11911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec66a752d944968bb5086e7940db27, State: Initialized, Role: FOLLOWER
I20260808 17:02:15.637125 11911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ec66a752d944968bb5086e7940db27" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40257 } }
I20260808 17:02:15.637205 11911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:15.637238 11911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:15.637286 11911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:15.637748 11911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ec66a752d944968bb5086e7940db27" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40257 } }
Aug 08 17:02:15 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:15.637841 11911 leader_election.cc:302] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22ec66a752d944968bb5086e7940db27; no voters:
I20260808 17:02:15.638007 11911 leader_election.cc:288] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:15.638062 11920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:15.638217 11920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [term 1 LEADER]: Becoming Leader. State: Replica: 22ec66a752d944968bb5086e7940db27, State: Running, Role: LEADER
I20260808 17:02:15.638258 11911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:15.638413 11920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ec66a752d944968bb5086e7940db27" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40257 } }
I20260808 17:02:15.638931 11921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22ec66a752d944968bb5086e7940db27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ec66a752d944968bb5086e7940db27" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40257 } } }
I20260808 17:02:15.639022 11921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:15.639225 11938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:15.639685 11922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22ec66a752d944968bb5086e7940db27. Latest consensus state: current_term: 1 leader_uuid: "22ec66a752d944968bb5086e7940db27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ec66a752d944968bb5086e7940db27" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40257 } } }
I20260808 17:02:15.640419 11922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:15.640663 11938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:15.641976 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:15.632134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.30.194.62:40257)
0808 17:02:15.632324 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:15.632334 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:15.632360 (+ 26us) server_negotiation.cc:407] Connection header received
0808 17:02:15.632759 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:15.632765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:15.632816 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:15.632929 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:15.633720 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.634761 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.635542 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.635945 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.636047 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:15.638342 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:15.638346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:15.638346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:15.638380 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:15.640258 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:15.640644 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:15.640648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:15.640648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:15.640749 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:15.640950 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:15.640951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:15.640952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:15.641249 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:15.641399 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:15.641647 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 17:02:15.641788 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260808 17:02:15.642679 11938 catalog_manager.cc:1347] Generated new cluster ID: 60f0e5cc23a74c0aac009c00ddfcaaa1
I20260808 17:02:15.642724 11938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:15.657900 11938 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:15.658579 11938 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:15.665833 11938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27: Generated new TSK 0
I20260808 17:02:15.666010 11938 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab.
Entry for principal oryx/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab.
I20260808 17:02:15.695889 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40257
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab
--principal=oryx/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:15.774296 11963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:15.774485 11963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:15.774508 11963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:15.775606 11963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:15.775642 11963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:15.775661 11963 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:02:15.775681 11963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:15.777118 11963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab
--principal=oryx/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.194.62:40257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:15.777299 11963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:15.777464 11963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:15.781983 11980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:15.782016 11976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:15.782116 11963 server_base.cc:1034] running on GCE node
W20260808 17:02:15.782167 11977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:15.782387 11963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:15.782610 11963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:15.783741 11963 hybrid_clock.cc:648] HybridClock initialized: now 1786208535783734 us; error 26 us; skew 500 ppm
Aug 08 17:02:15 dist-test-slave-2j7r krb5kdc[11764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208535, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:15.786108 11963 init.cc:377] Logged in from keytab as oryx/127.30.194.1@KRBTEST.COM (short username oryx)
I20260808 17:02:15.786482 11963 webserver.cc:460] Webserver started at http://127.30.194.1:42199/ using document root <none> and password file <none>
I20260808 17:02:15.786667 11963 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:15.786707 11963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:15.786813 11963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:15.787750 11963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "4bea28ee2dd54655aa483fec6f5ce503"
format_stamp: "Formatted at 2026-08-08 17:02:15 on dist-test-slave-2j7r"
server_key: "ff90e684e29b97667bfe76ce97e7fddf"
server_key_iv: "f22f49bc994b2ffa2ffdbb5b57463841"
server_key_version: "encryptionkey@0"
I20260808 17:02:15.788064 11963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "4bea28ee2dd54655aa483fec6f5ce503"
format_stamp: "Formatted at 2026-08-08 17:02:15 on dist-test-slave-2j7r"
server_key: "ff90e684e29b97667bfe76ce97e7fddf"
server_key_iv: "f22f49bc994b2ffa2ffdbb5b57463841"
server_key_version: "encryptionkey@0"
I20260808 17:02:15.789273 11963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:02:15.789940 11994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:15.790076 11963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:15.790156 11963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "4bea28ee2dd54655aa483fec6f5ce503"
format_stamp: "Formatted at 2026-08-08 17:02:15 on dist-test-slave-2j7r"
server_key: "ff90e684e29b97667bfe76ce97e7fddf"
server_key_iv: "f22f49bc994b2ffa2ffdbb5b57463841"
server_key_version: "encryptionkey@0"
I20260808 17:02:15.790213 11963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:15.803192 11963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:15.804840 11963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:15.804965 11963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:15.805415 11963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:15.805464 11963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:15.805490 11963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:15.805506 11963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:15.811348 11963 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:37163
I20260808 17:02:15.811424 12176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:37163 every 8 connection(s)
Aug 08 17:02:15 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208535, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:15.811664 11963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:02:15.814322 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:15.805799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:59509 (local address 127.30.194.62:40257)
0808 17:02:15.805838 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:15.805840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:15.806050 (+ 210us) server_negotiation.cc:407] Connection header received
0808 17:02:15.806481 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:15.806482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:15.806495 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:15.806543 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:15.807315 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.808069 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.809482 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.809795 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.809857 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:15.812140 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:15.812141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:15.812141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:15.812148 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:15.813302 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:15.813645 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:15.813646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:15.813646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:15.813690 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:15.813861 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:15.813861 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:15.813862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:15.813950 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:15.813992 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:15.814177 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 17:02:15.814259 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:02:15.814424 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:15.805803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40257 (local address 127.30.194.1:59509)
0808 17:02:15.805946 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:15.805977 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:15.806393 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:15.806620 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:15.806624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:15.806940 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:15.807199 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:15.807206 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.808157 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.808158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:15.809397 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:15.809400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.809883 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.809884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:15.810001 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:15.810409 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:15.810418 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:15.812035 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:15.813369 (+ 1334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:15.813377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:15.813379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:15.813585 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:15.813733 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:15.813733 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:15.813734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:15.813820 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:15.813993 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:15.813996 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:15.814095 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 17:02:15.814165 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 17:02:15.814272 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260808 17:02:15.814862 12181 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40257
I20260808 17:02:15.814937 12181 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:15.815085 12181 heartbeater.cc:511] Master 127.30.194.62:40257 requested a full tablet report, sending...
I20260808 17:02:15.815563 11838 ts_manager.cc:194] Registered new tserver with Master: 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163)
I20260808 17:02:15.816227 11838 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:59509
I20260808 17:02:15.820133 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 11963
I20260808 17:02:15.820245 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:02:15.820425 31496 external_mini_cluster.cc:1428] Setting key d5baccaec8b1bd4c51d45ce4bdcdd7f5
I20260808 17:02:15.820561 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:15.815754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:56813 (local address 127.30.194.62:40257)
0808 17:02:15.815786 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:15.815788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:15.815904 (+ 116us) server_negotiation.cc:407] Connection header received
0808 17:02:15.815956 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:15.815957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:15.815967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:15.816009 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:15.816353 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.816970 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.817682 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.817865 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.817912 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:15.818649 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:15.818650 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:15.818650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:15.818656 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:15.819614 (+ 958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:15.819888 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:15.819889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:15.819889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:15.819931 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:15.820105 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:15.820105 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:15.820105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:15.820184 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:15.820232 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:15.820413 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 17:02:15.820478 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 17:02:15.820570 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:15.815715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40257 (local address 127.30.194.1:56813)
0808 17:02:15.815842 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:15.815851 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:15.815904 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:15.816012 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:15.816013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:15.816207 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:15.816261 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:15.816263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.817051 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.817052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:15.817619 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:15.817622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.817916 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.817916 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:15.817959 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:15.818301 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:15.818306 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:15.818560 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:15.819674 (+ 1114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:15.819675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:15.819675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:15.819821 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:15.819987 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:15.819988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:15.819988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:15.820048 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:15.820275 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:15.820276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:15.820323 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 17:02:15.820387 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:02:15.820480 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for oryx/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab.
Entry for principal oryx/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab.
I20260808 17:02:15.843850 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40257
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab
--principal=oryx/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:15.923866 12191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:15.924044 12191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:15.924065 12191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:15.925168 12191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:15.925202 12191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:15.925221 12191 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:02:15.925237 12191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:15.926669 12191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab
--principal=oryx/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.194.62:40257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:15.926853 12191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:15.927021 12191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:15.931591 12210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:15.931648 12199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:15.931659 12191 server_base.cc:1034] running on GCE node
W20260808 17:02:15.931605 12197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:15.931998 12191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:15.932197 12191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:15.933323 12191 hybrid_clock.cc:648] HybridClock initialized: now 1786208535933315 us; error 27 us; skew 500 ppm
Aug 08 17:02:15 dist-test-slave-2j7r krb5kdc[11764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208535, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:15.935640 12191 init.cc:377] Logged in from keytab as oryx/127.30.194.2@KRBTEST.COM (short username oryx)
I20260808 17:02:15.936064 12191 webserver.cc:460] Webserver started at http://127.30.194.2:37057/ using document root <none> and password file <none>
I20260808 17:02:15.936268 12191 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:15.936311 12191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:15.936410 12191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:15.937379 12191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "70a07fb462614fb89e781dc0b18b1c03"
format_stamp: "Formatted at 2026-08-08 17:02:15 on dist-test-slave-2j7r"
server_key: "26b11b6ab1a8c440c919c227d1cc9ec2"
server_key_iv: "79304f7d3238574f5ae32c40fb687762"
server_key_version: "encryptionkey@0"
I20260808 17:02:15.937675 12191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "70a07fb462614fb89e781dc0b18b1c03"
format_stamp: "Formatted at 2026-08-08 17:02:15 on dist-test-slave-2j7r"
server_key: "26b11b6ab1a8c440c919c227d1cc9ec2"
server_key_iv: "79304f7d3238574f5ae32c40fb687762"
server_key_version: "encryptionkey@0"
I20260808 17:02:15.938928 12191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:15.939648 12223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:15.939821 12191 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:15.939917 12191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "70a07fb462614fb89e781dc0b18b1c03"
format_stamp: "Formatted at 2026-08-08 17:02:15 on dist-test-slave-2j7r"
server_key: "26b11b6ab1a8c440c919c227d1cc9ec2"
server_key_iv: "79304f7d3238574f5ae32c40fb687762"
server_key_version: "encryptionkey@0"
I20260808 17:02:15.939978 12191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:15.956995 12191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:15.958581 12191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:15.958701 12191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:15.959143 12191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:15.959179 12191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:15.959205 12191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:15.959223 12191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:02:15 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208535, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:15.965603 12191 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:35017
I20260808 17:02:15.965668 12418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:35017 every 8 connection(s)
I20260808 17:02:15.965955 12191 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:02:15.968088 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 12191
I20260808 17:02:15.968200 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:02:15.968340 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:15.959520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:48941 (local address 127.30.194.62:40257)
0808 17:02:15.959587 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:15.959590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:15.959751 (+ 161us) server_negotiation.cc:407] Connection header received
0808 17:02:15.960161 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:15.960163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:15.960175 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:15.960262 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:15.960964 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.961768 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.963216 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.963516 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.963589 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:15.965971 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:15.965972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:15.965973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:15.965980 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:15.967144 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:15.967493 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:15.967494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:15.967495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:15.967553 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:15.967762 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:15.967763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:15.967764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:15.967873 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:15.967930 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:15.968173 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 17:02:15.968251 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:02:15.968442 31496 external_mini_cluster.cc:1428] Setting key 0c9b31409b82ee6ae333e80dfbe6b4e8
I20260808 17:02:15.968480 12235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:15.959483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40257 (local address 127.30.194.2:48941)
0808 17:02:15.959634 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:15.959669 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:15.960081 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:15.960324 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:15.960329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:15.960619 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:15.960878 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:15.960885 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.961866 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.961867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:15.963100 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:15.963103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.963618 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.963619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:15.963772 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:15.964199 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:15.964207 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:15.965867 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:15.967213 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:15.967220 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:15.967222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:15.967427 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:15.967605 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:15.967606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:15.967606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:15.967707 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:15.967927 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:15.967933 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:15.968059 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 17:02:15.968143 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 17:02:15.968286 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260808 17:02:15.968935 12419 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40257
I20260808 17:02:15.969019 12419 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:15.969209 12419 heartbeater.cc:511] Master 127.30.194.62:40257 requested a full tablet report, sending...
I20260808 17:02:15.969585 11840 ts_manager.cc:194] Registered new tserver with Master: 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017)
I20260808 17:02:15.970023 11840 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:48941
I20260808 17:02:15.975065 12428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:15.969917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40257 (local address 127.30.194.2:45165)
0808 17:02:15.970049 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:15.970057 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:15.970188 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:15.970305 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:15.970306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:15.970471 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:15.970523 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:15.970524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.971164 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.971165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:15.971631 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:15.971632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.972019 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.972020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:15.972062 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:15.972403 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:15.972408 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:15.972654 (+ 246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:15.974165 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:15.974166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:15.974166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:15.974327 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:15.974521 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:15.974521 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:15.974521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:15.974569 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:15.974862 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:15.974863 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:15.974894 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 17:02:15.974936 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 17:02:15.975002 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260808 17:02:15.975112 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:15.969978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:45165 (local address 127.30.194.62:40257)
0808 17:02:15.970013 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:15.970015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:15.970105 (+ 90us) server_negotiation.cc:407] Connection header received
0808 17:02:15.970242 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:15.970243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:15.970252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:15.970292 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:15.970586 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.971089 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.971700 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:15.971954 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:15.972021 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:15.972736 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:15.972737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:15.972737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:15.972744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:15.974094 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:15.974404 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:15.974405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:15.974406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:15.974464 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:15.974623 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:15.974623 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:15.974623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:15.974803 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:15.974875 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:15.974964 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 17:02:15.975031 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for oryx/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab.
Entry for principal oryx/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab.
I20260808 17:02:15.992602 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40257
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab
--principal=oryx/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:16.071975 12435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:16.072154 12435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:16.072177 12435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:16.073312 12435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:16.073352 12435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:16.073374 12435 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:02:16.073402 12435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:16.074855 12435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab
--principal=oryx/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.194.62:40257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:16.075070 12435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:16.075273 12435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:02:16.079947 12435 server_base.cc:1034] running on GCE node
W20260808 17:02:16.079941 12448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:16.079960 12446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:16.079929 12444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:16.080327 12435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:16.080547 12435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:16.081678 12435 hybrid_clock.cc:648] HybridClock initialized: now 1786208536081664 us; error 26 us; skew 500 ppm
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208536, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:16.084013 12435 init.cc:377] Logged in from keytab as oryx/127.30.194.3@KRBTEST.COM (short username oryx)
I20260808 17:02:16.084412 12435 webserver.cc:460] Webserver started at http://127.30.194.3:39289/ using document root <none> and password file <none>
I20260808 17:02:16.084614 12435 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:16.084658 12435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:16.084776 12435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:16.085744 12435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "714ad573ef0d4d598eb0534352700fef"
format_stamp: "Formatted at 2026-08-08 17:02:16 on dist-test-slave-2j7r"
server_key: "b51f69bc45a82dc1c33627ba5c7ae7d2"
server_key_iv: "b90e871afd158d69ce762437f63d4c19"
server_key_version: "encryptionkey@0"
I20260808 17:02:16.086061 12435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "714ad573ef0d4d598eb0534352700fef"
format_stamp: "Formatted at 2026-08-08 17:02:16 on dist-test-slave-2j7r"
server_key: "b51f69bc45a82dc1c33627ba5c7ae7d2"
server_key_iv: "b90e871afd158d69ce762437f63d4c19"
server_key_version: "encryptionkey@0"
I20260808 17:02:16.087286 12435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:16.087992 12460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:16.088151 12435 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:16.088269 12435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "714ad573ef0d4d598eb0534352700fef"
format_stamp: "Formatted at 2026-08-08 17:02:16 on dist-test-slave-2j7r"
server_key: "b51f69bc45a82dc1c33627ba5c7ae7d2"
server_key_iv: "b90e871afd158d69ce762437f63d4c19"
server_key_version: "encryptionkey@0"
I20260808 17:02:16.088335 12435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:16.103255 12435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:16.105351 12435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:16.105487 12435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:16.105991 12435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:16.106033 12435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:16.106063 12435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:16.106084 12435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208536, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:16.113194 12435 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:37579
I20260808 17:02:16.113241 12652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:37579 every 8 connection(s)
I20260808 17:02:16.113574 12435 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:02:16.115803 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.106319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:47159 (local address 127.30.194.62:40257)
0808 17:02:16.106385 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.106387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.106555 (+ 168us) server_negotiation.cc:407] Connection header received
0808 17:02:16.106984 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.106985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.106999 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.107081 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:16.107751 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.108457 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.109707 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.109988 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.110068 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.112976 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:16.112978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:16.112978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:16.112986 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:16.114502 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:16.114887 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:16.114888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:16.114888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:16.114949 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:16.115184 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:16.115185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:16.115185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:16.115314 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:16.115372 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.115623 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.115716 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 17:02:16.115950 12478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.106293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40257 (local address 127.30.194.3:47159)
0808 17:02:16.106431 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.106468 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.106889 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.107127 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.107130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.107399 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:16.107650 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.107656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.108581 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.108582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.109606 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.109609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.110069 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.110070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.110205 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.110682 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:16.110692 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:16.112876 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:16.114573 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:16.114578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:16.114579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:16.114801 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:16.115015 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:16.115016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:16.115016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:16.115124 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:16.115381 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:16.115388 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:16.115507 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.115592 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.115731 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260808 17:02:16.116405 12653 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40257
I20260808 17:02:16.116482 12653 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:16.116647 12653 heartbeater.cc:511] Master 127.30.194.62:40257 requested a full tablet report, sending...
I20260808 17:02:16.116852 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 12435
I20260808 17:02:16.116990 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:02:16.117010 11838 ts_manager.cc:194] Registered new tserver with Master: 714ad573ef0d4d598eb0534352700fef (127.30.194.3:37579)
I20260808 17:02:16.117244 31496 external_mini_cluster.cc:1428] Setting key 9f3543966f8207ebe91c0d907650cdf8
I20260808 17:02:16.117482 11838 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:47159
I20260808 17:02:16.118216 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:02:16.123080 12661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.117466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40257 (local address 127.30.194.3:38845)
0808 17:02:16.117739 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.117750 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.117791 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.118047 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.118047 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.118231 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:16.118291 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.118292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.118898 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.118899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.119575 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.119578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.119928 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.119929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.119988 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.120333 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:16.120338 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:16.120631 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:16.122105 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:16.122106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:16.122106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:16.122278 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:16.122468 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:16.122469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:16.122469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:16.122520 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:16.122734 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:16.122735 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:16.122774 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.122878 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.122976 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":83,"threads_started":1}
I20260808 17:02:16.123128 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.117629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:38845 (local address 127.30.194.62:40257)
0808 17:02:16.117824 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.117826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.117832 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:02:16.117849 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.117849 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.117858 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.117910 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:16.118356 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.118799 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.119662 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.119857 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.119915 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.120707 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:16.120708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:16.120708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:16.120715 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:16.122021 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:16.122349 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:16.122349 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:16.122350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:16.122410 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:16.122568 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:16.122569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:16.122569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:16.122681 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:16.122743 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.122949 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.123049 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208536, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 17:02:16.140421 11918 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:16.135580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.30.194.62:40257)
0808 17:02:16.135634 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.135636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.135647 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:02:16.135668 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.135669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.135678 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.135723 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:16.136253 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.136694 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.137412 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.137585 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.137636 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.140353 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43736: BlockingRecv error: recv got EOF from 127.0.0.1:43736 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208536, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:16.146837 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.141107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43752 (local address 127.30.194.62:40257)
0808 17:02:16.141145 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.141147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.141177 (+ 30us) server_negotiation.cc:407] Connection header received
0808 17:02:16.141237 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.141238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.141245 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.141280 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:16.141674 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.142120 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.142813 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.142980 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.143026 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.144849 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:16.144850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:16.144850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:16.144855 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:16.145891 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:16.146237 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:16.146239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:16.146239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:16.146314 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:16.146487 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:16.146488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:16.146488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:16.146580 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:16.146648 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.146721 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.146780 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:02:16.148154 11840 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:02:16.149684 11840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:02:16.155570 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.151493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37163 (local address 127.0.0.1:55476)
0808 17:02:16.151635 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.151671 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.151734 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.152165 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.152168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.152173 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:16.152265 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.152267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.153562 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.153563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.154633 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.154634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.155280 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.155281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.155359 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.155373 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.155440 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.155490 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":1,"thread_start_us":54,"threads_started":1}
I20260808 17:02:16.155727 12691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.151639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45624 (local address 127.30.194.3:37579)
0808 17:02:16.152017 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.152020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.152030 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:16.152062 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.152064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.152076 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.152147 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:16.152571 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.153623 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.154609 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.155329 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.155404 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.155478 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.155512 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.155546 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":84,"threads_started":1}
I20260808 17:02:16.155763 12686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.151513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:35017 (local address 127.0.0.1:35462)
0808 17:02:16.152225 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.152236 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.152301 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.152457 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.152457 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.152459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:16.152507 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.152509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.153380 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.153381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.154602 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.154605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.155544 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.155545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.155607 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.155612 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.155660 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.155700 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":1,"thread_start_us":47,"threads_started":1}
I20260808 17:02:16.155831 12696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.151828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55476 (local address 127.30.194.1:37163)
0808 17:02:16.151983 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.151987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.151998 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:02:16.152040 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.152042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.152058 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.152147 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:16.152352 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.153483 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.154716 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.155195 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.155260 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.155341 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.155690 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.155733 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260808 17:02:16.155985 12693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.151695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.30.194.2:35017)
0808 17:02:16.151886 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.151890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.152301 (+ 411us) server_negotiation.cc:407] Connection header received
0808 17:02:16.152374 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.152376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.152391 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.152454 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:16.152552 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.153306 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.154685 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.155388 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.155444 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.155510 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.155851 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.155898 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260808 17:02:16.156180 12690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.151494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:37579 (local address 127.0.0.1:45624)
0808 17:02:16.151706 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.151717 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.151783 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.152165 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.152168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.152170 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:16.152492 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.152495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.153693 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.153694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.154538 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.154540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.155401 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.155402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.155461 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.155464 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.155499 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.155528 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":3,"thread_start_us":45,"threads_started":1}
I20260808 17:02:16.156471 12067 tablet_service.cc:1467] Processing CreateTablet for tablet fbd424d702df4492b68d0e375ed24d60 (DEFAULT_TABLE table=test-table [id=649ed97f1c2c4c5597e6fa525bbe3c6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:16.156599 12297 tablet_service.cc:1467] Processing CreateTablet for tablet fbd424d702df4492b68d0e375ed24d60 (DEFAULT_TABLE table=test-table [id=649ed97f1c2c4c5597e6fa525bbe3c6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:16.156831 12067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbd424d702df4492b68d0e375ed24d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.156819 12546 tablet_service.cc:1467] Processing CreateTablet for tablet fbd424d702df4492b68d0e375ed24d60 (DEFAULT_TABLE table=test-table [id=649ed97f1c2c4c5597e6fa525bbe3c6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:16.156946 12297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbd424d702df4492b68d0e375ed24d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.157161 12546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbd424d702df4492b68d0e375ed24d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.159854 12697 tablet_bootstrap.cc:492] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: Bootstrap starting.
I20260808 17:02:16.160434 12700 tablet_bootstrap.cc:492] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: Bootstrap starting.
I20260808 17:02:16.160602 12697 tablet_bootstrap.cc:654] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.160982 12697 log.cc:824] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:16.161031 12700 tablet_bootstrap.cc:654] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.161346 12702 tablet_bootstrap.cc:492] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: Bootstrap starting.
I20260808 17:02:16.161387 12700 log.cc:824] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:16.161724 12697 tablet_bootstrap.cc:492] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: No bootstrap required, opened a new log
I20260808 17:02:16.161790 12697 ts_tablet_manager.cc:1397] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:16.162098 12700 tablet_bootstrap.cc:492] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: No bootstrap required, opened a new log
I20260808 17:02:16.162169 12700 ts_tablet_manager.cc:1397] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:02:16.163662 12697 raft_consensus.cc:348] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } }
I20260808 17:02:16.163839 12697 raft_consensus.cc:374] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.163883 12697 raft_consensus.cc:729] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bea28ee2dd54655aa483fec6f5ce503, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.164007 12697 consensus_queue.cc:260] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } }
I20260808 17:02:16.164012 12700 raft_consensus.cc:348] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } }
I20260808 17:02:16.164165 12700 raft_consensus.cc:374] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.164208 12700 raft_consensus.cc:729] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714ad573ef0d4d598eb0534352700fef, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.164278 12697 ts_tablet_manager.cc:1428] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:02:16.164327 12700 consensus_queue.cc:260] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } }
I20260808 17:02:16.164409 12181 heartbeater.cc:503] Master 127.30.194.62:40257 was elected leader, sending a full tablet report...
I20260808 17:02:16.164589 12700 ts_tablet_manager.cc:1428] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 17:02:16.164614 12653 heartbeater.cc:503] Master 127.30.194.62:40257 was elected leader, sending a full tablet report...
I20260808 17:02:16.164939 12702 tablet_bootstrap.cc:654] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.165313 12702 log.cc:824] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:16.166183 12702 tablet_bootstrap.cc:492] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: No bootstrap required, opened a new log
I20260808 17:02:16.166241 12702 ts_tablet_manager.cc:1397] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260808 17:02:16.167603 12702 raft_consensus.cc:348] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } }
I20260808 17:02:16.167718 12702 raft_consensus.cc:374] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.167738 12702 raft_consensus.cc:729] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a07fb462614fb89e781dc0b18b1c03, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.167846 12702 consensus_queue.cc:260] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } }
I20260808 17:02:16.168067 12702 ts_tablet_manager.cc:1428] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:02:16.168139 12419 heartbeater.cc:503] Master 127.30.194.62:40257 was elected leader, sending a full tablet report...
W20260808 17:02:16.216454 12422 tablet.cc:2369] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:02:16.218819 12714 raft_consensus.cc:482] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:16.218915 12714 raft_consensus.cc:504] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } }
I20260808 17:02:16.219215 12714 leader_election.cc:288] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163), 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017)
I20260808 17:02:16.222038 12696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.219522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:43003 (local address 127.30.194.1:37163)
0808 17:02:16.219568 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.219570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.219650 (+ 80us) server_negotiation.cc:407] Connection header received
0808 17:02:16.219711 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.219712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.219720 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.219770 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:16.219917 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.220371 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.221382 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.221732 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.221793 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.221804 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.221932 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.221967 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:02:16.222065 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.219393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37163 (local address 127.30.194.3:43003)
0808 17:02:16.219581 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.219590 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.219654 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.219785 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.219786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.219788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:16.219859 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.219860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.220443 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.220444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.221303 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.221306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.221803 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.221804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.221873 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.221877 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.221917 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.221952 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":46,"thread_start_us":112,"threads_started":1}
I20260808 17:02:16.222055 12693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.219578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:35655 (local address 127.30.194.2:35017)
0808 17:02:16.219643 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.219645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.219654 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:16.219680 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.219681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.219690 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.219751 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:16.219913 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.220379 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.221380 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.221732 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.221803 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.221811 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.221959 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.221995 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:02:16.222088 12478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.219388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:35017 (local address 127.30.194.3:35655)
0808 17:02:16.219439 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.219450 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.219498 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.219767 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.219769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.219771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:16.219846 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.219848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.220452 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.220453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.221301 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.221304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.221809 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.221810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.221892 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.221895 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.221947 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.221987 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":38}
I20260808 17:02:16.222397 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd424d702df4492b68d0e375ed24d60" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bea28ee2dd54655aa483fec6f5ce503" is_pre_election: true
I20260808 17:02:16.222445 12341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd424d702df4492b68d0e375ed24d60" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a07fb462614fb89e781dc0b18b1c03" is_pre_election: true
I20260808 17:02:16.222527 12103 raft_consensus.cc:2454] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 0.
I20260808 17:02:16.222585 12341 raft_consensus.cc:2454] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 0.
I20260808 17:02:16.222750 12467 leader_election.cc:302] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bea28ee2dd54655aa483fec6f5ce503, 714ad573ef0d4d598eb0534352700fef; no voters:
I20260808 17:02:16.222862 12714 raft_consensus.cc:2781] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:16.222913 12714 raft_consensus.cc:482] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:16.222927 12714 raft_consensus.cc:3037] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:16.223817 12714 raft_consensus.cc:504] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } }
I20260808 17:02:16.223925 12714 leader_election.cc:288] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 election: Requested vote from peers 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163), 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017)
I20260808 17:02:16.224072 12341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd424d702df4492b68d0e375ed24d60" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a07fb462614fb89e781dc0b18b1c03"
I20260808 17:02:16.224146 12341 raft_consensus.cc:3037] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:16.224825 12341 raft_consensus.cc:2454] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 1.
I20260808 17:02:16.224920 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd424d702df4492b68d0e375ed24d60" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bea28ee2dd54655aa483fec6f5ce503"
I20260808 17:02:16.224989 12103 raft_consensus.cc:3037] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:16.224998 12465 leader_election.cc:302] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70a07fb462614fb89e781dc0b18b1c03, 714ad573ef0d4d598eb0534352700fef; no voters:
I20260808 17:02:16.225076 12714 raft_consensus.cc:2781] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:16.225231 12714 raft_consensus.cc:686] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 1 LEADER]: Becoming Leader. State: Replica: 714ad573ef0d4d598eb0534352700fef, State: Running, Role: LEADER
I20260808 17:02:16.225322 12714 consensus_queue.cc:237] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } }
I20260808 17:02:16.225778 12103 raft_consensus.cc:2454] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 1.
I20260808 17:02:16.226203 11838 catalog_manager.cc:5495] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef reported cstate change: term changed from 0 to 1, leader changed from <none> to 714ad573ef0d4d598eb0534352700fef (127.30.194.3). New cstate: current_term: 1 leader_uuid: "714ad573ef0d4d598eb0534352700fef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } health_report { overall_health: UNKNOWN } } }
I20260808 17:02:16.231271 12691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.229426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45636 (local address 127.30.194.3:37579)
0808 17:02:16.229476 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.229478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.229485 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:16.229527 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.229528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.229542 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.229595 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:16.229674 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.230079 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.230720 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.230893 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.230938 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.231030 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:16.231113 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:16.231161 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.231175 (+ 14us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.231189 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:02:16.232752 12341 raft_consensus.cc:1261] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Refusing update from remote peer 714ad573ef0d4d598eb0534352700fef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:16.232769 12103 raft_consensus.cc:1261] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Refusing update from remote peer 714ad573ef0d4d598eb0534352700fef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:16.233055 12714 consensus_queue.cc:1035] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:16.233208 12719 consensus_queue.cc:1035] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:16.235253 12722 mvcc.cc:205] Tried to move back new op lower bound from 7316310164407312384 to 7316310164379533312. Current Snapshot: MvccSnapshot[applied={T|T < 7316310164407312384}]
I20260808 17:02:16.235769 12724 mvcc.cc:205] Tried to move back new op lower bound from 7316310164407312384 to 7316310164379533312. Current Snapshot: MvccSnapshot[applied={T|T < 7316310164407312384}]
I20260808 17:02:16.235908 12723 mvcc.cc:205] Tried to move back new op lower bound from 7316310164407312384 to 7316310164379533312. Current Snapshot: MvccSnapshot[applied={T|T < 7316310164407312384}]
I20260808 17:02:16.237481 11840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:16.237555 11840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:16.238242 11840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27: Sending DeleteTablet for 3 replicas of tablet fbd424d702df4492b68d0e375ed24d60
I20260808 17:02:16.238554 12546 tablet_service.cc:1514] Processing DeleteTablet for tablet fbd424d702df4492b68d0e375ed24d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:16 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:45624
I20260808 17:02:16.238596 12067 tablet_service.cc:1514] Processing DeleteTablet for tablet fbd424d702df4492b68d0e375ed24d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:16 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:55476
I20260808 17:02:16.238741 12297 tablet_service.cc:1514] Processing DeleteTablet for tablet fbd424d702df4492b68d0e375ed24d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:16 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:35462
I20260808 17:02:16.238816 12738 tablet_replica.cc:330] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: stopping tablet replica
I20260808 17:02:16.238917 12738 raft_consensus.cc:2229] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:16.238967 12734 tablet_replica.cc:330] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: stopping tablet replica
I20260808 17:02:16.239089 12734 raft_consensus.cc:2229] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:02:16.239068 11840 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 17:02:16.239172 12734 raft_consensus.cc:2258] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 17:02:16.239192 11840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:02:16.239336 12738 raft_consensus.cc:2258] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:16.239601 12734 ts_tablet_manager.cc:1905] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:16.239725 12738 ts_tablet_manager.cc:1905] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:16.240697 12297 tablet_service.cc:1467] Processing CreateTablet for tablet bc618f22207e449fb1bea59954335a20 (DEFAULT_TABLE table=test-table [id=79332b8bde3f46379a4dd1b379f763fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:16.240801 12297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc618f22207e449fb1bea59954335a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.241207 12734 ts_tablet_manager.cc:1918] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:16.241268 12734 log.cc:1196] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/fbd424d702df4492b68d0e375ed24d60
I20260808 17:02:16.241340 12738 ts_tablet_manager.cc:1918] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:16.241405 12738 log.cc:1196] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/fbd424d702df4492b68d0e375ed24d60
I20260808 17:02:16.241498 12734 ts_tablet_manager.cc:1939] T fbd424d702df4492b68d0e375ed24d60 P 714ad573ef0d4d598eb0534352700fef: Deleting consensus metadata
I20260808 17:02:16.241650 12738 ts_tablet_manager.cc:1939] T fbd424d702df4492b68d0e375ed24d60 P 4bea28ee2dd54655aa483fec6f5ce503: Deleting consensus metadata
I20260808 17:02:16.242200 12702 tablet_bootstrap.cc:492] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03: Bootstrap starting.
I20260808 17:02:16.242408 11818 catalog_manager.cc:4841] TS 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163): tablet fbd424d702df4492b68d0e375ed24d60 (table test-table [id=649ed97f1c2c4c5597e6fa525bbe3c6b]) successfully deleted
I20260808 17:02:16.242635 12702 tablet_bootstrap.cc:654] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.242781 11814 catalog_manager.cc:4841] TS 714ad573ef0d4d598eb0534352700fef (127.30.194.3:37579): tablet fbd424d702df4492b68d0e375ed24d60 (table test-table [id=649ed97f1c2c4c5597e6fa525bbe3c6b]) successfully deleted
I20260808 17:02:16.242926 12546 tablet_service.cc:1467] Processing CreateTablet for tablet bc618f22207e449fb1bea59954335a20 (DEFAULT_TABLE table=test-table [id=79332b8bde3f46379a4dd1b379f763fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:16.243019 12546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc618f22207e449fb1bea59954335a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.243005 12067 tablet_service.cc:1467] Processing CreateTablet for tablet bc618f22207e449fb1bea59954335a20 (DEFAULT_TABLE table=test-table [id=79332b8bde3f46379a4dd1b379f763fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:16.243103 12067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc618f22207e449fb1bea59954335a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.244196 12700 tablet_bootstrap.cc:492] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: Bootstrap starting.
I20260808 17:02:16.244203 12697 tablet_bootstrap.cc:492] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: Bootstrap starting.
I20260808 17:02:16.244448 12702 tablet_bootstrap.cc:492] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03: No bootstrap required, opened a new log
I20260808 17:02:16.244489 12702 ts_tablet_manager.cc:1397] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:02:16.244617 12700 tablet_bootstrap.cc:654] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.244616 12697 tablet_bootstrap.cc:654] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.244638 12702 raft_consensus.cc:348] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.244697 12702 raft_consensus.cc:374] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.244716 12702 raft_consensus.cc:729] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a07fb462614fb89e781dc0b18b1c03, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.244753 12702 consensus_queue.cc:260] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.244830 12702 ts_tablet_manager.cc:1428] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:16.245317 12700 tablet_bootstrap.cc:492] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: No bootstrap required, opened a new log
I20260808 17:02:16.245349 12739 tablet_replica.cc:330] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: stopping tablet replica
I20260808 17:02:16.245365 12700 ts_tablet_manager.cc:1397] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 17:02:16.245445 12739 raft_consensus.cc:2229] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:16.245538 12739 raft_consensus.cc:2258] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:16.245499 12700 raft_consensus.cc:348] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.245575 12700 raft_consensus.cc:374] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.245600 12700 raft_consensus.cc:729] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714ad573ef0d4d598eb0534352700fef, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.245638 12700 consensus_queue.cc:260] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.245720 12700 ts_tablet_manager.cc:1428] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:16.246050 12739 ts_tablet_manager.cc:1905] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:16.246412 12697 tablet_bootstrap.cc:492] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: No bootstrap required, opened a new log
I20260808 17:02:16.246467 12697 ts_tablet_manager.cc:1397] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:16.246610 12697 raft_consensus.cc:348] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.246678 12697 raft_consensus.cc:374] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.246702 12697 raft_consensus.cc:729] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bea28ee2dd54655aa483fec6f5ce503, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.246744 12697 consensus_queue.cc:260] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.246831 12697 ts_tablet_manager.cc:1428] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:16.247753 12739 ts_tablet_manager.cc:1918] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:16.247818 12739 log.cc:1196] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/fbd424d702df4492b68d0e375ed24d60
I20260808 17:02:16.248018 12739 ts_tablet_manager.cc:1939] T fbd424d702df4492b68d0e375ed24d60 P 70a07fb462614fb89e781dc0b18b1c03: Deleting consensus metadata
I20260808 17:02:16.248425 11817 catalog_manager.cc:4841] TS 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017): tablet fbd424d702df4492b68d0e375ed24d60 (table test-table [id=649ed97f1c2c4c5597e6fa525bbe3c6b]) successfully deleted
W20260808 17:02:16.312359 12182 tablet.cc:2369] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:02:16.364244 12654 tablet.cc:2369] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:02:16.389866 12716 raft_consensus.cc:482] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:16.389951 12716 raft_consensus.cc:504] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.390264 12716 leader_election.cc:288] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163), 714ad573ef0d4d598eb0534352700fef (127.30.194.3:37579)
I20260808 17:02:16.393260 12696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.390489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:59477 (local address 127.30.194.1:37163)
0808 17:02:16.390563 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.390565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.390636 (+ 71us) server_negotiation.cc:407] Connection header received
0808 17:02:16.390707 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.390708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.390718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.390770 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:16.390929 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.391432 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.392545 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.392911 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.392966 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.392974 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.393155 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.393193 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 17:02:16.393325 12691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.390477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:43129 (local address 127.30.194.3:37579)
0808 17:02:16.390527 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.390529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.390542 (+ 13us) server_negotiation.cc:407] Connection header received
0808 17:02:16.390571 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.390572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.390582 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.390642 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:16.390791 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.391445 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.392545 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.392966 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.393037 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.393047 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.393213 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.393269 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:02:16.393312 12746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.390411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37163 (local address 127.30.194.2:59477)
0808 17:02:16.390560 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.390570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.390637 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.390788 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.390789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.390790 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:16.390871 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.390873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.391548 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.391550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.392459 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.392462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.392982 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.392983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.393053 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.393057 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.393145 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.393179 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":11,"thread_start_us":84,"threads_started":1}
I20260808 17:02:16.393313 12235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.390411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:37579 (local address 127.30.194.2:43129)
0808 17:02:16.390464 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.390475 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.390518 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.390652 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.390653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.390655 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:16.390729 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.390731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.391610 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.391611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.392463 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.392465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.393050 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.393051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.393138 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.393142 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.393197 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.393234 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":2}
I20260808 17:02:16.393589 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc618f22207e449fb1bea59954335a20" candidate_uuid: "70a07fb462614fb89e781dc0b18b1c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bea28ee2dd54655aa483fec6f5ce503" is_pre_election: true
I20260808 17:02:16.393672 12103 raft_consensus.cc:2454] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a07fb462614fb89e781dc0b18b1c03 in term 0.
I20260808 17:02:16.393767 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc618f22207e449fb1bea59954335a20" candidate_uuid: "70a07fb462614fb89e781dc0b18b1c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714ad573ef0d4d598eb0534352700fef" is_pre_election: true
I20260808 17:02:16.393857 12227 leader_election.cc:302] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bea28ee2dd54655aa483fec6f5ce503, 70a07fb462614fb89e781dc0b18b1c03; no voters:
I20260808 17:02:16.393900 12577 raft_consensus.cc:2454] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a07fb462614fb89e781dc0b18b1c03 in term 0.
I20260808 17:02:16.393973 12716 raft_consensus.cc:2781] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:16.394008 12716 raft_consensus.cc:482] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:16.394027 12716 raft_consensus.cc:3037] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:16.394677 12716 raft_consensus.cc:504] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.394791 12716 leader_election.cc:288] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [CANDIDATE]: Term 1 election: Requested vote from peers 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163), 714ad573ef0d4d598eb0534352700fef (127.30.194.3:37579)
I20260808 17:02:16.394919 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc618f22207e449fb1bea59954335a20" candidate_uuid: "70a07fb462614fb89e781dc0b18b1c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bea28ee2dd54655aa483fec6f5ce503"
I20260808 17:02:16.394948 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc618f22207e449fb1bea59954335a20" candidate_uuid: "70a07fb462614fb89e781dc0b18b1c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714ad573ef0d4d598eb0534352700fef"
I20260808 17:02:16.394995 12103 raft_consensus.cc:3037] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:16.395007 12577 raft_consensus.cc:3037] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:16.395717 12577 raft_consensus.cc:2454] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a07fb462614fb89e781dc0b18b1c03 in term 1.
I20260808 17:02:16.395717 12103 raft_consensus.cc:2454] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a07fb462614fb89e781dc0b18b1c03 in term 1.
I20260808 17:02:16.395874 12227 leader_election.cc:302] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bea28ee2dd54655aa483fec6f5ce503, 70a07fb462614fb89e781dc0b18b1c03; no voters:
I20260808 17:02:16.395963 12716 raft_consensus.cc:2781] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:16.396060 12716 raft_consensus.cc:686] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 LEADER]: Becoming Leader. State: Replica: 70a07fb462614fb89e781dc0b18b1c03, State: Running, Role: LEADER
I20260808 17:02:16.396121 12716 consensus_queue.cc:237] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.396788 11838 catalog_manager.cc:5495] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "70a07fb462614fb89e781dc0b18b1c03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } health_report { overall_health: UNKNOWN } } }
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208535, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.62@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:16.451977 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.445791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40257 (local address 127.0.0.1:43768)
0808 17:02:16.445988 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.445996 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.446055 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.446188 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.446188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.446498 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:16.446547 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.446549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.447161 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.447162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.447680 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.447681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.447954 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.447955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.447990 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.448494 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:16.448499 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:16.449977 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:16.451161 (+ 1184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:16.451168 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:16.451170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:16.451360 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:16.451493 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:16.451494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:16.451494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:16.451541 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:16.451702 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:16.451706 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:16.451753 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.451799 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.451883 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
I20260808 17:02:16.452021 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.445838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43768 (local address 127.30.194.62:40257)
0808 17:02:16.445900 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.445902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.446054 (+ 152us) server_negotiation.cc:407] Connection header received
0808 17:02:16.446100 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.446101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.446111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.446173 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:16.446610 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.447103 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.447733 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.447907 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.447949 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.450050 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:16.450052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:16.450052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:16.450058 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:16.451101 (+ 1043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:16.451416 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:16.451416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:16.451416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:16.451457 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:16.451575 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:16.451576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:16.451576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:16.451661 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:16.451704 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.451820 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.451925 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":34}
I20260808 17:02:16.453723 11838 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:43768:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 17:02:16.453912 11838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:02:16.455250 12546 tablet_service.cc:1467] Processing CreateTablet for tablet 1f0bd15709634fe1aabd9cd5ba65b983 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8f26f21f3a8b44d2bb97e70ef6a80017]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:02:16.455241 12067 tablet_service.cc:1467] Processing CreateTablet for tablet 1f0bd15709634fe1aabd9cd5ba65b983 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8f26f21f3a8b44d2bb97e70ef6a80017]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:02:16.455410 12546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f0bd15709634fe1aabd9cd5ba65b983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.455410 12067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f0bd15709634fe1aabd9cd5ba65b983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.456856 12700 tablet_bootstrap.cc:492] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: Bootstrap starting.
I20260808 17:02:16.456946 12297 tablet_service.cc:1467] Processing CreateTablet for tablet 1f0bd15709634fe1aabd9cd5ba65b983 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8f26f21f3a8b44d2bb97e70ef6a80017]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:02:16.457106 12297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f0bd15709634fe1aabd9cd5ba65b983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.457257 12700 tablet_bootstrap.cc:654] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.457489 12697 tablet_bootstrap.cc:492] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: Bootstrap starting.
I20260808 17:02:16.457809 12700 tablet_bootstrap.cc:492] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: No bootstrap required, opened a new log
I20260808 17:02:16.457857 12700 ts_tablet_manager.cc:1397] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:16.457890 12697 tablet_bootstrap.cc:654] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.457962 12700 raft_consensus.cc:348] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.458026 12700 raft_consensus.cc:374] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.458045 12700 raft_consensus.cc:729] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714ad573ef0d4d598eb0534352700fef, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.458084 12700 consensus_queue.cc:260] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.458169 12700 ts_tablet_manager.cc:1428] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:16.458298 12702 tablet_bootstrap.cc:492] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: Bootstrap starting.
I20260808 17:02:16.458283 12719 tablet_replica.cc:439] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } }
I20260808 17:02:16.458348 12719 tablet_replica.cc:442] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:02:16.458559 12697 tablet_bootstrap.cc:492] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: No bootstrap required, opened a new log
I20260808 17:02:16.458614 12697 ts_tablet_manager.cc:1397] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:16.458644 12702 tablet_bootstrap.cc:654] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.458748 12697 raft_consensus.cc:348] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.458817 12697 raft_consensus.cc:374] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.458838 12697 raft_consensus.cc:729] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bea28ee2dd54655aa483fec6f5ce503, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.458877 12697 consensus_queue.cc:260] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.458974 12697 ts_tablet_manager.cc:1428] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:16.459101 12709 tablet_replica.cc:439] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } }
I20260808 17:02:16.459190 12709 tablet_replica.cc:442] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:02:16.459311 12702 tablet_bootstrap.cc:492] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: No bootstrap required, opened a new log
I20260808 17:02:16.459363 12702 ts_tablet_manager.cc:1397] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 17:02:16.459501 12702 raft_consensus.cc:348] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.459569 12702 raft_consensus.cc:374] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.459589 12702 raft_consensus.cc:729] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a07fb462614fb89e781dc0b18b1c03, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.459631 12702 consensus_queue.cc:260] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.459725 12702 ts_tablet_manager.cc:1428] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:16.459832 12716 tablet_replica.cc:439] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } }
I20260808 17:02:16.459914 12716 tablet_replica.cc:442] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:02:16.659108 12719 raft_consensus.cc:482] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:16.659196 12719 raft_consensus.cc:504] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.659329 12719 leader_election.cc:288] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163), 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017)
I20260808 17:02:16.659514 12341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f0bd15709634fe1aabd9cd5ba65b983" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a07fb462614fb89e781dc0b18b1c03" is_pre_election: true
I20260808 17:02:16.659615 12341 raft_consensus.cc:2454] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 0.
I20260808 17:02:16.659574 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f0bd15709634fe1aabd9cd5ba65b983" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bea28ee2dd54655aa483fec6f5ce503" is_pre_election: true
I20260808 17:02:16.659660 12103 raft_consensus.cc:2454] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 0.
I20260808 17:02:16.659785 12465 leader_election.cc:302] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70a07fb462614fb89e781dc0b18b1c03, 714ad573ef0d4d598eb0534352700fef; no voters:
I20260808 17:02:16.659880 12719 raft_consensus.cc:2781] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:16.659924 12719 raft_consensus.cc:482] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:16.659938 12719 raft_consensus.cc:3037] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:16.660581 12719 raft_consensus.cc:504] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.660696 12719 leader_election.cc:288] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 election: Requested vote from peers 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163), 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017)
I20260808 17:02:16.660842 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f0bd15709634fe1aabd9cd5ba65b983" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bea28ee2dd54655aa483fec6f5ce503"
I20260808 17:02:16.660856 12341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f0bd15709634fe1aabd9cd5ba65b983" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a07fb462614fb89e781dc0b18b1c03"
I20260808 17:02:16.660912 12103 raft_consensus.cc:3037] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:16.660928 12341 raft_consensus.cc:3037] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:16.661536 12103 raft_consensus.cc:2454] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 1.
I20260808 17:02:16.661586 12341 raft_consensus.cc:2454] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 1.
I20260808 17:02:16.661715 12467 leader_election.cc:302] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bea28ee2dd54655aa483fec6f5ce503, 714ad573ef0d4d598eb0534352700fef; no voters:
I20260808 17:02:16.661799 12719 raft_consensus.cc:2781] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:16.661842 12719 raft_consensus.cc:686] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [term 1 LEADER]: Becoming Leader. State: Replica: 714ad573ef0d4d598eb0534352700fef, State: Running, Role: LEADER
I20260808 17:02:16.661891 12719 consensus_queue.cc:237] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.662142 12714 tablet_replica.cc:439] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: TxnStatusTablet state changed. Reason: New leader 714ad573ef0d4d598eb0534352700fef. Latest consensus state: current_term: 1 leader_uuid: "714ad573ef0d4d598eb0534352700fef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } }
I20260808 17:02:16.662279 12714 tablet_replica.cc:442] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:02:16.662462 12763 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 17:02:16.662529 11840 catalog_manager.cc:5495] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef reported cstate change: term changed from 0 to 1, leader changed from <none> to 714ad573ef0d4d598eb0534352700fef (127.30.194.3). New cstate: current_term: 1 leader_uuid: "714ad573ef0d4d598eb0534352700fef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } health_report { overall_health: HEALTHY } } }
I20260808 17:02:16.662613 12763 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 17:02:16.684206 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.681717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:37579 (local address 127.0.0.1:41622)
0808 17:02:16.681787 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.681800 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.681852 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.682000 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.682000 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.682001 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:02:16.682045 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.682047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.682890 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.682891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.683439 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.683441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.683878 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.683878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.683919 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.683934 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:02:16.684085 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:02:16.684089 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.684118 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.684154 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 17:02:16.684213 12691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.681771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41622 (local address 127.30.194.3:37579)
0808 17:02:16.681820 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.681823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.681833 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:16.681922 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.681923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.681933 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.681995 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:16.682079 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.682816 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.683507 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.683806 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.683876 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.683997 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:16.684037 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:16.684086 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.684122 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.684141 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:02:16.685036 12103 raft_consensus.cc:1261] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Refusing update from remote peer 714ad573ef0d4d598eb0534352700fef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:16.685077 12341 raft_consensus.cc:1261] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Refusing update from remote peer 714ad573ef0d4d598eb0534352700fef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:16.685256 12714 consensus_queue.cc:1035] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:16.685343 12719 consensus_queue.cc:1035] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:16.685842 12709 tablet_replica.cc:439] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: TxnStatusTablet state changed. Reason: New leader 714ad573ef0d4d598eb0534352700fef. Latest consensus state: current_term: 1 leader_uuid: "714ad573ef0d4d598eb0534352700fef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } }
I20260808 17:02:16.685927 12709 tablet_replica.cc:442] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:02:16.685904 12716 tablet_replica.cc:439] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: TxnStatusTablet state changed. Reason: New leader 714ad573ef0d4d598eb0534352700fef. Latest consensus state: current_term: 1 leader_uuid: "714ad573ef0d4d598eb0534352700fef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } }
I20260808 17:02:16.686002 12716 tablet_replica.cc:442] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:02:16.686551 12714 tablet_replica.cc:439] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "714ad573ef0d4d598eb0534352700fef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } }
I20260808 17:02:16.686630 12714 tablet_replica.cc:442] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:02:16.686789 12714 tablet_replica.cc:439] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "714ad573ef0d4d598eb0534352700fef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } }
I20260808 17:02:16.686861 12714 tablet_replica.cc:442] T 1f0bd15709634fe1aabd9cd5ba65b983 P 714ad573ef0d4d598eb0534352700fef: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:02:16.686996 12709 tablet_replica.cc:439] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "714ad573ef0d4d598eb0534352700fef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } }
I20260808 17:02:16.687072 12709 tablet_replica.cc:442] T 1f0bd15709634fe1aabd9cd5ba65b983 P 4bea28ee2dd54655aa483fec6f5ce503: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:02:16.688020 12716 tablet_replica.cc:439] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "714ad573ef0d4d598eb0534352700fef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } } }
I20260808 17:02:16.688099 12716 tablet_replica.cc:442] T 1f0bd15709634fe1aabd9cd5ba65b983 P 70a07fb462614fb89e781dc0b18b1c03: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:02:16.691129 12693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.688658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.30.194.2:35017)
0808 17:02:16.688694 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.688695 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.688706 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:02:16.688789 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.688790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.688808 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.688865 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:16.688960 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.689699 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.690344 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.690638 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.690700 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.690832 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:16.690963 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:16.691030 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.691045 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.691063 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:02:16.695483 12691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.693407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:48699 (local address 127.30.194.3:37579)
0808 17:02:16.693442 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.693444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.693557 (+ 113us) server_negotiation.cc:407] Connection header received
0808 17:02:16.693631 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.693632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.693640 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.693683 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:16.693777 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.694197 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.694880 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.695055 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.695106 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.695283 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:16.695313 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:16.695353 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.695406 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.695423 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:02:16.695681 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.693360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:37579 (local address 127.30.194.2:48699)
0808 17:02:16.693503 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.693514 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.693588 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.693684 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.693684 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.693685 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:02:16.693735 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.693736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.694265 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.694266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.694809 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.694810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.695158 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.695159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.695204 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.695216 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:02:16.695364 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:02:16.695366 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.695405 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.695443 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":93,"threads_started":1}
I20260808 17:02:16.697855 12103 raft_consensus.cc:1261] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Refusing update from remote peer 70a07fb462614fb89e781dc0b18b1c03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:16.697985 12577 raft_consensus.cc:1261] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Refusing update from remote peer 70a07fb462614fb89e781dc0b18b1c03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:16.698079 12716 consensus_queue.cc:1035] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:16.698212 12747 consensus_queue.cc:1035] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:16.705888 12693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.702932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:60077 (local address 127.30.194.2:35017)
0808 17:02:16.702967 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.702969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.703125 (+ 156us) server_negotiation.cc:407] Connection header received
0808 17:02:16.703164 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.703164 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.703173 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.703224 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:16.703400 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.704077 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.705188 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.705481 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.705516 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.705711 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:16.705737 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:16.705775 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.705827 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.705842 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:02:16.705964 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.702899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:35017 (local address 127.30.194.3:60077)
0808 17:02:16.703045 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:16.703055 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:16.703115 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:16.703264 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:16.703266 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:16.703281 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:02:16.703329 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.703330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.704166 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.704167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.705069 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:16.705071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.705568 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.705569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:16.705627 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.705640 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:02:16.705778 (+ 138us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:02:16.705780 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:02:16.705825 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:02:16.705866 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260808 17:02:16.711819 11840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:16.711898 11840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:16.712424 11840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27: Sending DeleteTablet for 3 replicas of tablet bc618f22207e449fb1bea59954335a20
I20260808 17:02:16.712641 12297 tablet_service.cc:1514] Processing DeleteTablet for tablet bc618f22207e449fb1bea59954335a20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:16 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:35462
I20260808 17:02:16.712724 12739 tablet_replica.cc:330] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03: stopping tablet replica
I20260808 17:02:16.712767 12739 raft_consensus.cc:2229] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:02:16.712774 12067 tablet_service.cc:1514] Processing DeleteTablet for tablet bc618f22207e449fb1bea59954335a20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:16 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:55476
I20260808 17:02:16.712849 12739 raft_consensus.cc:2258] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:16.712865 12738 tablet_replica.cc:330] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: stopping tablet replica
I20260808 17:02:16.712929 12738 raft_consensus.cc:2229] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:16.712970 12738 raft_consensus.cc:2258] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:16.713275 12738 ts_tablet_manager.cc:1905] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:16.713295 12739 ts_tablet_manager.cc:1905] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:16.714807 12738 ts_tablet_manager.cc:1918] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:02:16.714854 12738 log.cc:1196] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/bc618f22207e449fb1bea59954335a20
I20260808 17:02:16.714841 12739 ts_tablet_manager.cc:1918] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:02:16.714890 12739 log.cc:1196] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/bc618f22207e449fb1bea59954335a20
I20260808 17:02:16.715091 12738 ts_tablet_manager.cc:1939] T bc618f22207e449fb1bea59954335a20 P 4bea28ee2dd54655aa483fec6f5ce503: Deleting consensus metadata
I20260808 17:02:16.715098 12739 ts_tablet_manager.cc:1939] T bc618f22207e449fb1bea59954335a20 P 70a07fb462614fb89e781dc0b18b1c03: Deleting consensus metadata
I20260808 17:02:16.715409 11817 catalog_manager.cc:4841] TS 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017): tablet bc618f22207e449fb1bea59954335a20 (table test-table [id=79332b8bde3f46379a4dd1b379f763fa]) successfully deleted
I20260808 17:02:16.716367 11818 catalog_manager.cc:4841] TS 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163): tablet bc618f22207e449fb1bea59954335a20 (table test-table [id=79332b8bde3f46379a4dd1b379f763fa]) successfully deleted
I20260808 17:02:16.716570 12546 tablet_service.cc:1514] Processing DeleteTablet for tablet bc618f22207e449fb1bea59954335a20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:16 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:45624
I20260808 17:02:16.716651 12734 tablet_replica.cc:330] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: stopping tablet replica
I20260808 17:02:16.716699 12734 raft_consensus.cc:2229] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:16.716735 12734 raft_consensus.cc:2258] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:16.717043 12734 ts_tablet_manager.cc:1905] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:16.718251 12734 ts_tablet_manager.cc:1918] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:02:16.718302 12734 log.cc:1196] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/bc618f22207e449fb1bea59954335a20
I20260808 17:02:16.718479 12734 ts_tablet_manager.cc:1939] T bc618f22207e449fb1bea59954335a20 P 714ad573ef0d4d598eb0534352700fef: Deleting consensus metadata
I20260808 17:02:16.718737 11814 catalog_manager.cc:4841] TS 714ad573ef0d4d598eb0534352700fef (127.30.194.3:37579): tablet bc618f22207e449fb1bea59954335a20 (table test-table [id=79332b8bde3f46379a4dd1b379f763fa]) successfully deleted
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 17:02:16.742617 11918 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:16.737427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51580 (local address 127.30.194.62:40257)
0808 17:02:16.737471 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.737473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.737596 (+ 123us) server_negotiation.cc:407] Connection header received
0808 17:02:16.737655 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.737656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.737667 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.737711 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:16.738276 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.738810 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.739652 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.739835 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.739892 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.742542 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51580: BlockingRecv error: recv got EOF from 127.0.0.1:51580 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 17:02:16 dist-test-slave-2j7r krb5kdc[11764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM
I20260808 17:02:16.749854 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:16.743731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.30.194.62:40257)
0808 17:02:16.743770 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:16.743771 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:16.743925 (+ 154us) server_negotiation.cc:407] Connection header received
0808 17:02:16.743980 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:16.743981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:16.743990 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:16.744034 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:16.744446 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.744890 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.745570 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:16.745761 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:16.745812 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:16.747651 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:16.747653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:16.747653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:16.747660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:16.748847 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:16.749195 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:16.749196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:16.749197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:16.749253 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:16.749419 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:16.749419 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:16.749420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:16.749521 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:16.749602 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:16.749703 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 17:02:16.749772 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:02:16.751189 11840 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:02:16.751341 11840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:02:16.752646 12067 tablet_service.cc:1467] Processing CreateTablet for tablet cedacf3a18bd463495d37e811fd890a2 (DEFAULT_TABLE table=test-table [id=f6f67a760a1449deafff9458e3aac7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:16.752780 12067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cedacf3a18bd463495d37e811fd890a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.753793 12697 tablet_bootstrap.cc:492] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503: Bootstrap starting.
I20260808 17:02:16.753834 12546 tablet_service.cc:1467] Processing CreateTablet for tablet cedacf3a18bd463495d37e811fd890a2 (DEFAULT_TABLE table=test-table [id=f6f67a760a1449deafff9458e3aac7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:16.753938 12546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cedacf3a18bd463495d37e811fd890a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.754190 12697 tablet_bootstrap.cc:654] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.755033 12700 tablet_bootstrap.cc:492] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef: Bootstrap starting.
I20260808 17:02:16.755352 12700 tablet_bootstrap.cc:654] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.755666 12297 tablet_service.cc:1467] Processing CreateTablet for tablet cedacf3a18bd463495d37e811fd890a2 (DEFAULT_TABLE table=test-table [id=f6f67a760a1449deafff9458e3aac7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:16.755772 12297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cedacf3a18bd463495d37e811fd890a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:16.756623 12702 tablet_bootstrap.cc:492] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03: Bootstrap starting.
I20260808 17:02:16.757027 12702 tablet_bootstrap.cc:654] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:16.757040 12700 tablet_bootstrap.cc:492] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef: No bootstrap required, opened a new log
I20260808 17:02:16.757328 12697 tablet_bootstrap.cc:492] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503: No bootstrap required, opened a new log
I20260808 17:02:16.757372 12697 ts_tablet_manager.cc:1397] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 17:02:16.757534 12697 raft_consensus.cc:348] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.757601 12697 raft_consensus.cc:374] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.757622 12697 raft_consensus.cc:729] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bea28ee2dd54655aa483fec6f5ce503, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.757663 12697 consensus_queue.cc:260] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.757746 12697 ts_tablet_manager.cc:1428] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:16.757936 12702 tablet_bootstrap.cc:492] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03: No bootstrap required, opened a new log
I20260808 17:02:16.757980 12702 ts_tablet_manager.cc:1397] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:16.757077 12700 ts_tablet_manager.cc:1397] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:16.758122 12702 raft_consensus.cc:348] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.758183 12702 raft_consensus.cc:374] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.758200 12702 raft_consensus.cc:729] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a07fb462614fb89e781dc0b18b1c03, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.758193 12700 raft_consensus.cc:348] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.758253 12700 raft_consensus.cc:374] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:16.758239 12702 consensus_queue.cc:260] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.758272 12700 raft_consensus.cc:729] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714ad573ef0d4d598eb0534352700fef, State: Initialized, Role: FOLLOWER
I20260808 17:02:16.758317 12702 ts_tablet_manager.cc:1428] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:16.758318 12700 consensus_queue.cc:260] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:16.758401 12700 ts_tablet_manager.cc:1428] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:17.058133 12775 raft_consensus.cc:482] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:17.058233 12775 raft_consensus.cc:504] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.058401 12775 leader_election.cc:288] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163), 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017)
I20260808 17:02:17.058619 12341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cedacf3a18bd463495d37e811fd890a2" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a07fb462614fb89e781dc0b18b1c03" is_pre_election: true
I20260808 17:02:17.058612 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cedacf3a18bd463495d37e811fd890a2" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bea28ee2dd54655aa483fec6f5ce503" is_pre_election: true
I20260808 17:02:17.058707 12103 raft_consensus.cc:2454] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 0.
I20260808 17:02:17.058709 12341 raft_consensus.cc:2454] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 0.
I20260808 17:02:17.058876 12467 leader_election.cc:302] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bea28ee2dd54655aa483fec6f5ce503, 714ad573ef0d4d598eb0534352700fef; no voters:
I20260808 17:02:17.058979 12775 raft_consensus.cc:2781] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:17.059021 12775 raft_consensus.cc:482] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:17.059041 12775 raft_consensus.cc:3037] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:17.059741 12775 raft_consensus.cc:504] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.059859 12775 leader_election.cc:288] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 election: Requested vote from peers 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163), 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017)
I20260808 17:02:17.060035 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cedacf3a18bd463495d37e811fd890a2" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bea28ee2dd54655aa483fec6f5ce503"
I20260808 17:02:17.060046 12341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cedacf3a18bd463495d37e811fd890a2" candidate_uuid: "714ad573ef0d4d598eb0534352700fef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a07fb462614fb89e781dc0b18b1c03"
I20260808 17:02:17.060104 12103 raft_consensus.cc:3037] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:17.060120 12341 raft_consensus.cc:3037] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:17.060659 12103 raft_consensus.cc:2454] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 1.
I20260808 17:02:17.060814 12341 raft_consensus.cc:2454] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 714ad573ef0d4d598eb0534352700fef in term 1.
I20260808 17:02:17.060834 12467 leader_election.cc:302] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bea28ee2dd54655aa483fec6f5ce503, 714ad573ef0d4d598eb0534352700fef; no voters:
I20260808 17:02:17.060979 12775 raft_consensus.cc:2781] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:17.061025 12775 raft_consensus.cc:686] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 1 LEADER]: Becoming Leader. State: Replica: 714ad573ef0d4d598eb0534352700fef, State: Running, Role: LEADER
I20260808 17:02:17.061069 12775 consensus_queue.cc:237] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.061627 11837 catalog_manager.cc:5495] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef reported cstate change: term changed from 0 to 1, leader changed from <none> to 714ad573ef0d4d598eb0534352700fef (127.30.194.3). New cstate: current_term: 1 leader_uuid: "714ad573ef0d4d598eb0534352700fef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } health_report { overall_health: HEALTHY } } }
I20260808 17:02:17.109225 12691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:17.107036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.30.194.3:37579)
0808 17:02:17.107091 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:17.107093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:17.107102 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:17.107127 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:17.107128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:17.107139 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:17.107187 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:17.107346 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:17.107833 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:17.108554 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:17.108761 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:17.108816 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:17.108956 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:17.108994 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:17.109038 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:17.109126 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 17:02:17.109145 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:02:17.110036 12341 raft_consensus.cc:1261] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Refusing update from remote peer 714ad573ef0d4d598eb0534352700fef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:17.110037 12103 raft_consensus.cc:1261] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Refusing update from remote peer 714ad573ef0d4d598eb0534352700fef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:17.110276 12714 consensus_queue.cc:1035] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:17.110399 12775 consensus_queue.cc:1035] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:17.112773 11838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:17.112885 11838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:17.113456 11838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27: Sending DeleteTablet for 3 replicas of tablet cedacf3a18bd463495d37e811fd890a2
I20260808 17:02:17.113664 12067 tablet_service.cc:1514] Processing DeleteTablet for tablet cedacf3a18bd463495d37e811fd890a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:17 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:55476
I20260808 17:02:17.113727 12297 tablet_service.cc:1514] Processing DeleteTablet for tablet cedacf3a18bd463495d37e811fd890a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:17 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:35462
I20260808 17:02:17.113756 12546 tablet_service.cc:1514] Processing DeleteTablet for tablet cedacf3a18bd463495d37e811fd890a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:17 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:45624
I20260808 17:02:17.113831 12734 tablet_replica.cc:330] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef: stopping tablet replica
I20260808 17:02:17.113833 12738 tablet_replica.cc:330] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503: stopping tablet replica
I20260808 17:02:17.113822 12739 tablet_replica.cc:330] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03: stopping tablet replica
I20260808 17:02:17.113880 12738 raft_consensus.cc:2229] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:17.113886 12734 raft_consensus.cc:2229] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:02:17.113888 12739 raft_consensus.cc:2229] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:17.113926 12738 raft_consensus.cc:2258] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:17.113929 12739 raft_consensus.cc:2258] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:17.113965 12734 raft_consensus.cc:2258] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:17.114251 12739 ts_tablet_manager.cc:1905] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:17.114411 12738 ts_tablet_manager.cc:1905] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:17.115724 12734 ts_tablet_manager.cc:1905] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:17.115736 12739 ts_tablet_manager.cc:1918] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:17.115787 12739 log.cc:1196] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/cedacf3a18bd463495d37e811fd890a2
I20260808 17:02:17.115986 12739 ts_tablet_manager.cc:1939] T cedacf3a18bd463495d37e811fd890a2 P 70a07fb462614fb89e781dc0b18b1c03: Deleting consensus metadata
I20260808 17:02:17.116288 11817 catalog_manager.cc:4841] TS 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017): tablet cedacf3a18bd463495d37e811fd890a2 (table test-table [id=f6f67a760a1449deafff9458e3aac7b2]) successfully deleted
I20260808 17:02:17.116943 12734 ts_tablet_manager.cc:1918] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:17.117003 12734 log.cc:1196] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/cedacf3a18bd463495d37e811fd890a2
I20260808 17:02:17.117220 12734 ts_tablet_manager.cc:1939] T cedacf3a18bd463495d37e811fd890a2 P 714ad573ef0d4d598eb0534352700fef: Deleting consensus metadata
I20260808 17:02:17.117317 12738 ts_tablet_manager.cc:1918] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:17.117364 12738 log.cc:1196] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/cedacf3a18bd463495d37e811fd890a2
I20260808 17:02:17.117344 11838 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:02:17.117483 11838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:02:17.117571 12738 ts_tablet_manager.cc:1939] T cedacf3a18bd463495d37e811fd890a2 P 4bea28ee2dd54655aa483fec6f5ce503: Deleting consensus metadata
I20260808 17:02:17.117573 11814 catalog_manager.cc:4841] TS 714ad573ef0d4d598eb0534352700fef (127.30.194.3:37579): tablet cedacf3a18bd463495d37e811fd890a2 (table test-table [id=f6f67a760a1449deafff9458e3aac7b2]) successfully deleted
I20260808 17:02:17.117864 11818 catalog_manager.cc:4841] TS 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163): tablet cedacf3a18bd463495d37e811fd890a2 (table test-table [id=f6f67a760a1449deafff9458e3aac7b2]) successfully deleted
I20260808 17:02:17.118579 12067 tablet_service.cc:1467] Processing CreateTablet for tablet f4527ad058b843848934431f0cf8826b (DEFAULT_TABLE table=test-table [id=4e7ae11cdef643db99067229ca89f86a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:17.118597 12297 tablet_service.cc:1467] Processing CreateTablet for tablet f4527ad058b843848934431f0cf8826b (DEFAULT_TABLE table=test-table [id=4e7ae11cdef643db99067229ca89f86a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:17.118687 12067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4527ad058b843848934431f0cf8826b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:17.118687 12297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4527ad058b843848934431f0cf8826b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:17.119782 12702 tablet_bootstrap.cc:492] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03: Bootstrap starting.
I20260808 17:02:17.119777 12697 tablet_bootstrap.cc:492] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503: Bootstrap starting.
I20260808 17:02:17.119954 12546 tablet_service.cc:1467] Processing CreateTablet for tablet f4527ad058b843848934431f0cf8826b (DEFAULT_TABLE table=test-table [id=4e7ae11cdef643db99067229ca89f86a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:17.120062 12546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4527ad058b843848934431f0cf8826b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:17.120085 12702 tablet_bootstrap.cc:654] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:17.120151 12697 tablet_bootstrap.cc:654] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:17.120682 12702 tablet_bootstrap.cc:492] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03: No bootstrap required, opened a new log
I20260808 17:02:17.120699 12697 tablet_bootstrap.cc:492] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503: No bootstrap required, opened a new log
I20260808 17:02:17.120743 12697 ts_tablet_manager.cc:1397] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:17.120744 12702 ts_tablet_manager.cc:1397] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:17.120849 12702 raft_consensus.cc:348] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.120879 12697 raft_consensus.cc:348] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.120914 12702 raft_consensus.cc:374] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:17.120934 12702 raft_consensus.cc:729] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a07fb462614fb89e781dc0b18b1c03, State: Initialized, Role: FOLLOWER
I20260808 17:02:17.120932 12697 raft_consensus.cc:374] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:17.120958 12697 raft_consensus.cc:729] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bea28ee2dd54655aa483fec6f5ce503, State: Initialized, Role: FOLLOWER
I20260808 17:02:17.120977 12702 consensus_queue.cc:260] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.120996 12697 consensus_queue.cc:260] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.121062 12702 ts_tablet_manager.cc:1428] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:17.121074 12697 ts_tablet_manager.cc:1428] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:17.121234 12700 tablet_bootstrap.cc:492] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef: Bootstrap starting.
I20260808 17:02:17.121496 12700 tablet_bootstrap.cc:654] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:17.122027 12700 tablet_bootstrap.cc:492] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef: No bootstrap required, opened a new log
I20260808 17:02:17.122073 12700 ts_tablet_manager.cc:1397] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 17:02:17.122157 12700 raft_consensus.cc:348] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.122219 12700 raft_consensus.cc:374] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:17.122246 12700 raft_consensus.cc:729] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714ad573ef0d4d598eb0534352700fef, State: Initialized, Role: FOLLOWER
I20260808 17:02:17.122293 12700 consensus_queue.cc:260] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.122383 12700 ts_tablet_manager.cc:1428] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:17.248582 12779 raft_consensus.cc:482] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:17.248679 12779 raft_consensus.cc:504] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.248844 12779 leader_election.cc:288] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163), 714ad573ef0d4d598eb0534352700fef (127.30.194.3:37579)
I20260808 17:02:17.249053 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4527ad058b843848934431f0cf8826b" candidate_uuid: "70a07fb462614fb89e781dc0b18b1c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bea28ee2dd54655aa483fec6f5ce503" is_pre_election: true
I20260808 17:02:17.249053 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4527ad058b843848934431f0cf8826b" candidate_uuid: "70a07fb462614fb89e781dc0b18b1c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714ad573ef0d4d598eb0534352700fef" is_pre_election: true
I20260808 17:02:17.249159 12103 raft_consensus.cc:2454] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a07fb462614fb89e781dc0b18b1c03 in term 0.
I20260808 17:02:17.249174 12577 raft_consensus.cc:2454] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a07fb462614fb89e781dc0b18b1c03 in term 0.
I20260808 17:02:17.249323 12227 leader_election.cc:302] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bea28ee2dd54655aa483fec6f5ce503, 70a07fb462614fb89e781dc0b18b1c03; no voters:
I20260808 17:02:17.249465 12779 raft_consensus.cc:2781] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:17.249521 12779 raft_consensus.cc:482] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:17.249542 12779 raft_consensus.cc:3037] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:17.250289 12779 raft_consensus.cc:504] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.250420 12779 leader_election.cc:288] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [CANDIDATE]: Term 1 election: Requested vote from peers 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163), 714ad573ef0d4d598eb0534352700fef (127.30.194.3:37579)
I20260808 17:02:17.250576 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4527ad058b843848934431f0cf8826b" candidate_uuid: "70a07fb462614fb89e781dc0b18b1c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714ad573ef0d4d598eb0534352700fef"
I20260808 17:02:17.250599 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4527ad058b843848934431f0cf8826b" candidate_uuid: "70a07fb462614fb89e781dc0b18b1c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bea28ee2dd54655aa483fec6f5ce503"
I20260808 17:02:17.250658 12577 raft_consensus.cc:3037] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:17.250663 12103 raft_consensus.cc:3037] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:17.251389 12577 raft_consensus.cc:2454] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a07fb462614fb89e781dc0b18b1c03 in term 1.
I20260808 17:02:17.251389 12103 raft_consensus.cc:2454] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a07fb462614fb89e781dc0b18b1c03 in term 1.
I20260808 17:02:17.251559 12227 leader_election.cc:302] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bea28ee2dd54655aa483fec6f5ce503, 70a07fb462614fb89e781dc0b18b1c03; no voters:
I20260808 17:02:17.251648 12779 raft_consensus.cc:2781] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:17.251744 12779 raft_consensus.cc:686] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 1 LEADER]: Becoming Leader. State: Replica: 70a07fb462614fb89e781dc0b18b1c03, State: Running, Role: LEADER
I20260808 17:02:17.251794 12779 consensus_queue.cc:237] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } }
I20260808 17:02:17.252364 11837 catalog_manager.cc:5495] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "70a07fb462614fb89e781dc0b18b1c03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70a07fb462614fb89e781dc0b18b1c03" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 35017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 } health_report { overall_health: UNKNOWN } } }
I20260808 17:02:17.266220 12834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:17.264075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51474 (local address 127.30.194.2:35017)
0808 17:02:17.264177 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:17.264180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:17.264186 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:02:17.264206 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:17.264206 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:17.264218 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:17.264272 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:17.264384 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:17.264855 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:17.265596 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:17.265802 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:17.265855 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:17.266002 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:17.266038 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:17.266080 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:17.266128 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 17:02:17.266149 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":51,"threads_started":1}
I20260808 17:02:17.268014 12577 raft_consensus.cc:1261] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Refusing update from remote peer 70a07fb462614fb89e781dc0b18b1c03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:17.268055 12103 raft_consensus.cc:1261] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Refusing update from remote peer 70a07fb462614fb89e781dc0b18b1c03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:17.268214 12779 consensus_queue.cc:1035] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bea28ee2dd54655aa483fec6f5ce503" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:17.268316 12830 consensus_queue.cc:1035] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "714ad573ef0d4d598eb0534352700fef" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:17.278143 11838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:17.278245 11838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:17.278800 11838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22ec66a752d944968bb5086e7940db27: Sending DeleteTablet for 3 replicas of tablet f4527ad058b843848934431f0cf8826b
I20260808 17:02:17.279011 12067 tablet_service.cc:1514] Processing DeleteTablet for tablet f4527ad058b843848934431f0cf8826b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:17 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:55476
I20260808 17:02:17.279044 12546 tablet_service.cc:1514] Processing DeleteTablet for tablet f4527ad058b843848934431f0cf8826b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:17 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:45624
I20260808 17:02:17.279098 12738 tablet_replica.cc:330] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503: stopping tablet replica
I20260808 17:02:17.279081 12297 tablet_service.cc:1514] Processing DeleteTablet for tablet f4527ad058b843848934431f0cf8826b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:17 UTC) from {username='oryx', principal='oryx/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:35462
I20260808 17:02:17.279129 12734 tablet_replica.cc:330] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef: stopping tablet replica
I20260808 17:02:17.279140 12738 raft_consensus.cc:2229] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:17.279173 12738 raft_consensus.cc:2258] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:17.279184 12734 raft_consensus.cc:2229] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:17.279222 12734 raft_consensus.cc:2258] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:17.279237 12739 tablet_replica.cc:330] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03: stopping tablet replica
I20260808 17:02:17.279277 12739 raft_consensus.cc:2229] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:02:17.279321 12739 raft_consensus.cc:2258] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:17.279541 12738 ts_tablet_manager.cc:1905] T f4527ad058b843848934431f0cf8826b P 4bea28ee2dd54655aa483fec6f5ce503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:17.279531 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 11963
I20260808 17:02:17.279649 12739 ts_tablet_manager.cc:1905] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:17.279879 12734 ts_tablet_manager.cc:1905] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:17.281122 12739 ts_tablet_manager.cc:1918] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:02:17.281172 12739 log.cc:1196] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/f4527ad058b843848934431f0cf8826b
I20260808 17:02:17.281239 12734 ts_tablet_manager.cc:1918] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:02:17.281282 12734 log.cc:1196] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/f4527ad058b843848934431f0cf8826b
I20260808 17:02:17.281378 12739 ts_tablet_manager.cc:1939] T f4527ad058b843848934431f0cf8826b P 70a07fb462614fb89e781dc0b18b1c03: Deleting consensus metadata
I20260808 17:02:17.281451 12734 ts_tablet_manager.cc:1939] T f4527ad058b843848934431f0cf8826b P 714ad573ef0d4d598eb0534352700fef: Deleting consensus metadata
I20260808 17:02:17.284600 11817 catalog_manager.cc:4841] TS 70a07fb462614fb89e781dc0b18b1c03 (127.30.194.2:35017): tablet f4527ad058b843848934431f0cf8826b (table test-table [id=4e7ae11cdef643db99067229ca89f86a]) successfully deleted
I20260808 17:02:17.291729 11814 catalog_manager.cc:4841] TS 714ad573ef0d4d598eb0534352700fef (127.30.194.3:37579): tablet f4527ad058b843848934431f0cf8826b (table test-table [id=4e7ae11cdef643db99067229ca89f86a]) successfully deleted
W20260808 17:02:17.294186 11818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:37163 (error 108)
I20260808 17:02:17.294507 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 12191
I20260808 17:02:17.297420 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:17.294534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37163 (local address 127.0.0.1:35578)
0808 17:02:17.297314 (+ 2780us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:17.297366 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2760,"thread_start_us":43,"threads_started":1}
W20260808 17:02:17.297514 11818 catalog_manager.cc:4584] TS 4bea28ee2dd54655aa483fec6f5ce503 (127.30.194.1:37163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4527ad058b843848934431f0cf8826b: Network error: Client connection negotiation failed: client connection to 127.30.194.1:37163: connect: Connection refused (error 111)
I20260808 17:02:17.308072 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 12435
I20260808 17:02:17.317337 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 11787
2026-08-08T17:02:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3065 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:02:17 dist-test-slave-2j7r krb5kdc[12851](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:02:17 dist-test-slave-2j7r krb5kdc[12851](info): set up 2 sockets
Aug 08 17:02:17 dist-test-slave-2j7r krb5kdc[12851](info): commencing operation
krb5kdc: starting...
W20260808 17:02:18.460498 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:02:18 dist-test-slave-2j7r krb5kdc[12851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:02:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:02:18.567778 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40661
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:40661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:18.648473 12873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:18.648664 12873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:18.648696 12873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:18.649811 12873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:18.649856 12873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:18.649871 12873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:18.649884 12873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:18.649897 12873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:18.651327 12873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:40661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:18.651561 12873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:18.651777 12873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:18.656260 12879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:18.656306 12886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:18.656250 12882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:18.656399 12873 server_base.cc:1034] running on GCE node
I20260808 17:02:18.656669 12873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:18.656973 12873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:18.658120 12873 hybrid_clock.cc:648] HybridClock initialized: now 1786208538658101 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T17:02:18Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1344 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:02:18 dist-test-slave-2j7r krb5kdc[12893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:02:18 dist-test-slave-2j7r krb5kdc[12893](info): set up 2 sockets
Aug 08 17:02:18 dist-test-slave-2j7r krb5kdc[12893](info): commencing operation
krb5kdc: starting...
W20260808 17:02:19.807902 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:02:19 dist-test-slave-2j7r krb5kdc[12893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:02:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:02:19.923507 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:35419
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:35419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:20.002038 12911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:20.002215 12911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:20.002235 12911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:20.003325 12911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:20.003365 12911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:20.003377 12911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:20.003401 12911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:20.003424 12911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:20.004896 12911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:35419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:35419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:20.005152 12911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:20.005385 12911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:20.010067 12924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.010119 12911 server_base.cc:1034] running on GCE node
W20260808 17:02:20.010100 12926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:20.010100 12928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.010505 12911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:20.010773 12911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:20.011914 12911 hybrid_clock.cc:648] HybridClock initialized: now 1786208540011897 us; error 33 us; skew 500 ppm
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208540, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:20.014238 12911 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:02:20.014619 12911 webserver.cc:460] Webserver started at http://127.30.194.62:36717/ using document root <none> and password file <none>
I20260808 17:02:20.014802 12911 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:20.014848 12911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:20.014983 12911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:20.015947 12911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "6d63dd80a821452f96547ec8dbcdec36"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "ca656c1b01d6343362ce6242323518fe"
server_key_iv: "89f59f3bed0c1dbb324027e14c263dec"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.016260 12911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "6d63dd80a821452f96547ec8dbcdec36"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "ca656c1b01d6343362ce6242323518fe"
server_key_iv: "89f59f3bed0c1dbb324027e14c263dec"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.017503 12911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:02:20.018127 12942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.018270 12911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:20.018353 12911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "6d63dd80a821452f96547ec8dbcdec36"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "ca656c1b01d6343362ce6242323518fe"
server_key_iv: "89f59f3bed0c1dbb324027e14c263dec"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.018414 12911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:20.029206 12911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:20.031159 12911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:20.031280 12911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:20.034852 12911 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:35419
I20260808 17:02:20.034893 13025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:35419 every 8 connection(s)
I20260808 17:02:20.035169 12911 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:20.035938 13027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:20.037437 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 12911
I20260808 17:02:20.037532 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:20.037688 31496 external_mini_cluster.cc:1428] Setting key e04f46312bfc1e1948e44868181f32d4
I20260808 17:02:20.038122 13027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36: Bootstrap starting.
I20260808 17:02:20.038682 13027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:20.038982 13027 log.cc:824] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:20.039671 13027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36: No bootstrap required, opened a new log
I20260808 17:02:20.041674 13027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } }
I20260808 17:02:20.041797 13027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:20.041837 13027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d63dd80a821452f96547ec8dbcdec36, State: Initialized, Role: FOLLOWER
I20260808 17:02:20.041953 13027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } }
I20260808 17:02:20.042037 13027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:20.042068 13027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:20.042116 13027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:20.042724 13027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } }
I20260808 17:02:20.042850 13027 leader_election.cc:302] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d63dd80a821452f96547ec8dbcdec36; no voters:
I20260808 17:02:20.043020 13027 leader_election.cc:288] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:20.043058 13035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:20.043251 13027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:20.043252 13035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 1 LEADER]: Becoming Leader. State: Replica: 6d63dd80a821452f96547ec8dbcdec36, State: Running, Role: LEADER
I20260808 17:02:20.044456 13035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } }
I20260808 17:02:20.045001 13036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d63dd80a821452f96547ec8dbcdec36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } } }
I20260808 17:02:20.045228 13036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:20.045074 13037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d63dd80a821452f96547ec8dbcdec36. Latest consensus state: current_term: 1 leader_uuid: "6d63dd80a821452f96547ec8dbcdec36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } } }
I20260808 17:02:20.045441 13037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:20.045432 13061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:20.045981 13061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:20.047389 13061 catalog_manager.cc:1347] Generated new cluster ID: e19f2b431de24955b90ac23944d2d051
I20260808 17:02:20.047441 13061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:20.047993 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:20.038161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57060 (local address 127.30.194.62:35419)
0808 17:02:20.038302 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:20.038309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:20.038332 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:02:20.038719 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:20.038725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:20.038771 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:20.038902 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:20.039583 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.040450 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.041525 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.041937 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.042013 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.044341 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:20.044345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:20.044345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:20.044357 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:20.046297 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:20.046707 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:20.046709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:20.046710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:20.046812 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:20.047029 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:20.047030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:20.047030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:20.047322 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:20.047464 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:20.047719 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 17:02:20.047846 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260808 17:02:20.072499 13061 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:20.073231 13061 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:20.082581 13061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36: Generated new TSK 0
I20260808 17:02:20.082739 13061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:02:20.105692 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35419
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:20.184530 13069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:20.184710 13069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:20.184731 13069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:20.185842 13069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:20.185889 13069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:20.185914 13069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:20.187265 13069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:20.187433 13069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:20.187595 13069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:20.192113 13077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:20.192168 13081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:20.192180 13079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.192301 13069 server_base.cc:1034] running on GCE node
I20260808 17:02:20.192579 13069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:20.192793 13069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:20.193920 13069 hybrid_clock.cc:648] HybridClock initialized: now 1786208540193911 us; error 27 us; skew 500 ppm
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208540, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:20.196116 13069 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:02:20.196506 13069 webserver.cc:460] Webserver started at http://127.30.194.1:35007/ using document root <none> and password file <none>
I20260808 17:02:20.196673 13069 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:20.196710 13069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:20.196797 13069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:20.197743 13069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "94c415fb180b42d0bd8954badb4730ef"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "16dbfa1ac939e560f2c3d6f6dddf1c4d"
server_key_iv: "9d0578793bf0b852a1fcfabf865b1a2c"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.198037 13069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "94c415fb180b42d0bd8954badb4730ef"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "16dbfa1ac939e560f2c3d6f6dddf1c4d"
server_key_iv: "9d0578793bf0b852a1fcfabf865b1a2c"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.199168 13069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:20.199798 13095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.199949 13069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:20.200032 13069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "94c415fb180b42d0bd8954badb4730ef"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "16dbfa1ac939e560f2c3d6f6dddf1c4d"
server_key_iv: "9d0578793bf0b852a1fcfabf865b1a2c"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.200089 13069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:20.227029 13069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:20.228709 13069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:20.228840 13069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:20.229065 13069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:20.229410 13069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:20.229445 13069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.229475 13069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:20.229501 13069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.235023 13069 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:41065
I20260808 17:02:20.235064 13277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:41065 every 8 connection(s)
I20260808 17:02:20.235363 13069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:02:20.239823 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 13069
I20260808 17:02:20.239929 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:02:20.240108 31496 external_mini_cluster.cc:1428] Setting key 3cf1d030e313cf4ad8e9fcdcf7f53667
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208540, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:20.244269 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:20.236073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:46751 (local address 127.30.194.62:35419)
0808 17:02:20.236151 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:20.236154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:20.236309 (+ 155us) server_negotiation.cc:407] Connection header received
0808 17:02:20.236792 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:20.236793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:20.236802 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:20.236851 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:20.237806 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.238321 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.239282 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.239468 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.239509 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.241624 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:20.241626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:20.241626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:20.241633 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:20.242847 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:20.243266 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:20.243268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:20.243268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:20.243323 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:20.243603 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:20.243604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:20.243605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:20.243749 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:20.243807 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:20.244093 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 17:02:20.244163 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 17:02:20.244462 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:20.236039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:46751)
0808 17:02:20.236181 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:20.236227 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:20.236714 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:20.236905 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:20.236908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:20.237526 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:20.237730 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.237735 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.238416 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.238417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.239212 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.239215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.239518 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.239519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.239612 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.240134 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:20.240142 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:20.241537 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:20.242931 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:20.242939 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:20.242941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:20.243205 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:20.243387 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:20.243388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:20.243388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:20.243538 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:20.243812 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:20.243819 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:20.243973 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 17:02:20.244082 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 17:02:20.244241 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260808 17:02:20.244951 13278 heartbeater.cc:347] Connected to a master server at 127.30.194.62:35419
I20260808 17:02:20.245040 13278 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:20.245242 13278 heartbeater.cc:511] Master 127.30.194.62:35419 requested a full tablet report, sending...
I20260808 17:02:20.245747 12963 ts_manager.cc:194] Registered new tserver with Master: 94c415fb180b42d0bd8954badb4730ef (127.30.194.1:41065)
I20260808 17:02:20.246344 12963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:46751
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:02:20.263765 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35419
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:20.341662 13287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:20.341841 13287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:20.341862 13287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:20.342926 13287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:20.342958 13287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:20.342983 13287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:20.344379 13287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:20.344590 13287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:20.344787 13287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:20.349330 13298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.349395 13287 server_base.cc:1034] running on GCE node
W20260808 17:02:20.349365 13295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:20.349344 13294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.349759 13287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:20.349987 13287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:20.351120 13287 hybrid_clock.cc:648] HybridClock initialized: now 1786208540351109 us; error 28 us; skew 500 ppm
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208540, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:20.353475 13287 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:02:20.353880 13287 webserver.cc:460] Webserver started at http://127.30.194.2:45535/ using document root <none> and password file <none>
I20260808 17:02:20.354044 13287 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:20.354080 13287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:20.354167 13287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:20.355053 13287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "b6e54f9174d64eb9ad210b70abbf7830"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "7cac3a624d2fa36d954fe6412b90df0d"
server_key_iv: "2f162e3d0e59cef4e2c3a59b8cc79410"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.355321 13287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "b6e54f9174d64eb9ad210b70abbf7830"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "7cac3a624d2fa36d954fe6412b90df0d"
server_key_iv: "2f162e3d0e59cef4e2c3a59b8cc79410"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.356451 13287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:02:20.357021 13308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.357178 13287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.357280 13287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "b6e54f9174d64eb9ad210b70abbf7830"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "7cac3a624d2fa36d954fe6412b90df0d"
server_key_iv: "2f162e3d0e59cef4e2c3a59b8cc79410"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.357331 13287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:20.363822 13287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:20.365334 13287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:20.365429 13287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:20.365584 13287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:20.365803 13287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:20.365831 13287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.365849 13287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:20.365865 13287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.371016 13287 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:46205
I20260808 17:02:20.371079 13493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:46205 every 8 connection(s)
I20260808 17:02:20.371311 13287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208540, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:20.377971 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 13287
I20260808 17:02:20.378088 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:02:20.378264 31496 external_mini_cluster.cc:1428] Setting key 5686104867058947bf65cc6b01baf527
I20260808 17:02:20.379379 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:20.371924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:54925 (local address 127.30.194.62:35419)
0808 17:02:20.371970 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:20.371972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:20.372136 (+ 164us) server_negotiation.cc:407] Connection header received
0808 17:02:20.372562 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:20.372564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:20.372573 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:20.372608 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:20.373470 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.373946 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.374821 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.375002 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.375053 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.376769 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:20.376770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:20.376770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:20.376777 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:20.378074 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:20.378455 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:20.378456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:20.378457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:20.378525 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:20.378745 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:20.378746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:20.378746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:20.378883 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:20.378938 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:20.379255 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 17:02:20.379316 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:02:20.379555 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:20.371893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:54925)
0808 17:02:20.372036 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:20.372066 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:20.372478 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:20.372660 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:20.372663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:20.373223 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:20.373401 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.373405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.374006 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.374007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.374749 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.374751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.375060 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.375060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.375153 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.375498 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:20.375505 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:20.376684 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:20.378146 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:20.378152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:20.378154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:20.378384 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:20.378587 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:20.378587 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:20.378588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:20.378689 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:20.378948 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:20.378955 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:20.379104 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 17:02:20.379209 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 17:02:20.379345 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260808 17:02:20.379992 13494 heartbeater.cc:347] Connected to a master server at 127.30.194.62:35419
I20260808 17:02:20.380069 13494 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:20.380223 13494 heartbeater.cc:511] Master 127.30.194.62:35419 requested a full tablet report, sending...
I20260808 17:02:20.380558 12963 ts_manager.cc:194] Registered new tserver with Master: b6e54f9174d64eb9ad210b70abbf7830 (127.30.194.2:46205)
I20260808 17:02:20.380905 12963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:54925
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:02:20.400848 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35419
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:20.478565 13500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:20.478753 13500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:20.478785 13500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:20.479880 13500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:20.479918 13500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:20.479951 13500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:20.481315 13500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:35419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:20.481535 13500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:20.481734 13500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:20.486297 13511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:20.486289 13508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:20.486290 13507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.486620 13500 server_base.cc:1034] running on GCE node
I20260808 17:02:20.486809 13500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:20.487004 13500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:20.488157 13500 hybrid_clock.cc:648] HybridClock initialized: now 1786208540488144 us; error 28 us; skew 500 ppm
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208540, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:20.490388 13500 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:02:20.490797 13500 webserver.cc:460] Webserver started at http://127.30.194.3:34901/ using document root <none> and password file <none>
I20260808 17:02:20.490979 13500 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:20.491024 13500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:20.491137 13500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:20.492060 13500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "50a54411a9b24302bc089a8dacd65816"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "45eb1ca19557723445819111198b61aa"
server_key_iv: "3cf8aa425c529d1181d8604fe4b0cf16"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.492362 13500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "50a54411a9b24302bc089a8dacd65816"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "45eb1ca19557723445819111198b61aa"
server_key_iv: "3cf8aa425c529d1181d8604fe4b0cf16"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.493588 13500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:02:20.494213 13523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.494379 13500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:20.494454 13500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "50a54411a9b24302bc089a8dacd65816"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "45eb1ca19557723445819111198b61aa"
server_key_iv: "3cf8aa425c529d1181d8604fe4b0cf16"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.494519 13500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:20.515976 13500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:20.517771 13500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:20.517884 13500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:20.518082 13500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:20.518353 13500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:20.518395 13500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.518421 13500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:20.518438 13500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.523912 13500 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:34185
I20260808 17:02:20.523974 13707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:34185 every 8 connection(s)
I20260808 17:02:20.524230 13500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:02:20.524744 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 13500
I20260808 17:02:20.524869 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:02:20.525051 31496 external_mini_cluster.cc:1428] Setting key 6fc1368bbf7d581e6fabbb3b33a14b80
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208540, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:20.532580 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:20.524869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:35033 (local address 127.30.194.62:35419)
0808 17:02:20.524927 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:20.524930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:20.525154 (+ 224us) server_negotiation.cc:407] Connection header received
0808 17:02:20.525763 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:20.525764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:20.525776 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:20.525825 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:20.526729 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.527218 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.528143 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.528347 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.528400 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.530328 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:20.530329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:20.530330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:20.530336 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:20.531471 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:20.531810 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:20.531811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:20.531812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:20.531856 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:20.532042 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:20.532043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:20.532043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:20.532158 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:20.532208 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:20.532439 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 17:02:20.532515 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:02:20.532665 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:20.524836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:35033)
0808 17:02:20.525005 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:20.525049 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:20.525671 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:20.525888 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:20.525892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:20.526462 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:20.526631 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.526635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.527300 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.527301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.528065 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.528068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.528413 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.528415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.528539 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.528904 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:20.528912 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:20.530247 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:20.531539 (+ 1292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:20.531546 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:20.531548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:20.531753 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:20.531902 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:20.531903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:20.531903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:20.531993 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:20.532205 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:20.532208 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:20.532311 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 17:02:20.532392 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:02:20.532503 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260808 17:02:20.533165 13708 heartbeater.cc:347] Connected to a master server at 127.30.194.62:35419
I20260808 17:02:20.533286 13708 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:20.533454 13708 heartbeater.cc:511] Master 127.30.194.62:35419 requested a full tablet report, sending...
I20260808 17:02:20.533792 12963 ts_manager.cc:194] Registered new tserver with Master: 50a54411a9b24302bc089a8dacd65816 (127.30.194.3:34185)
I20260808 17:02:20.534158 12963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:35033
I20260808 17:02:20.536579 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab.
Entry for principal oryx/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab.
WARNING: no policy specified for oryx/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab.
Entry for principal oryx/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab.
WARNING: no policy specified for oryx/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab.
Entry for principal oryx/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab.
I20260808 17:02:20.597146 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 13069
I20260808 17:02:20.607364 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 13287
I20260808 17:02:20.622067 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 13500
I20260808 17:02:20.633870 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 12911
I20260808 17:02:20.645668 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:35419
--webserver_interface=127.30.194.62
--webserver_port=36717
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:35419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:20.724134 13721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:20.724318 13721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:20.724339 13721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:20.725401 13721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:20.725432 13721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:20.725445 13721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:20.725455 13721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:20.725469 13721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:20.726856 13721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:35419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:35419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=36717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:20.727028 13721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:20.727187 13721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:20.731669 13731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:20.731817 13729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:20.731686 13728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.731907 13721 server_base.cc:1034] running on GCE node
I20260808 17:02:20.732134 13721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:20.732388 13721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:20.733518 13721 hybrid_clock.cc:648] HybridClock initialized: now 1786208540733505 us; error 28 us; skew 500 ppm
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208540, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:20.735800 13721 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:02:20.736194 13721 webserver.cc:460] Webserver started at http://127.30.194.62:36717/ using document root <none> and password file <none>
I20260808 17:02:20.736379 13721 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:20.736440 13721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:20.737694 13721 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:02:20.738286 13740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.738431 13721 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:20.738511 13721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "6d63dd80a821452f96547ec8dbcdec36"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "ca656c1b01d6343362ce6242323518fe"
server_key_iv: "89f59f3bed0c1dbb324027e14c263dec"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.738790 13721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:20.748843 13721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:20.750751 13721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:20.750882 13721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:20.754422 13721 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:35419
I20260808 17:02:20.754458 13814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:35419 every 8 connection(s)
I20260808 17:02:20.754768 13721 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:20.756047 13815 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:02:20.756634 13815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36: Bootstrap starting.
I20260808 17:02:20.759850 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 13721
I20260808 17:02:20.760196 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:41065
--local_ip_for_outbound_sockets=127.30.194.1
--tserver_master_addrs=127.30.194.62:35419
--webserver_port=35007
--webserver_interface=127.30.194.1
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.194.1
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:02:20.764925 13815 log.cc:824] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:20.766490 13815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:02:20.766739 13815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36: Bootstrap complete.
I20260808 17:02:20.768692 13815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } }
I20260808 17:02:20.768961 13815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d63dd80a821452f96547ec8dbcdec36, State: Initialized, Role: FOLLOWER
I20260808 17:02:20.769120 13815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } }
I20260808 17:02:20.769184 13815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:20.769209 13815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:20.769253 13815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:02:20.770145 13815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } }
I20260808 17:02:20.770229 13815 leader_election.cc:302] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d63dd80a821452f96547ec8dbcdec36; no voters:
I20260808 17:02:20.770376 13815 leader_election.cc:288] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 17:02:20.770426 13824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:02:20.770612 13824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [term 2 LEADER]: Becoming Leader. State: Replica: 6d63dd80a821452f96547ec8dbcdec36, State: Running, Role: LEADER
I20260808 17:02:20.770622 13815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:20.770767 13824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } }
I20260808 17:02:20.771054 13825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6d63dd80a821452f96547ec8dbcdec36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } } }
I20260808 17:02:20.771124 13825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:20.771332 13826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d63dd80a821452f96547ec8dbcdec36. Latest consensus state: current_term: 2 leader_uuid: "6d63dd80a821452f96547ec8dbcdec36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d63dd80a821452f96547ec8dbcdec36" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35419 } } }
I20260808 17:02:20.771400 13826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:20.771504 13833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:20.771987 13833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:20.772163 13833 catalog_manager.cc:1259] Loaded cluster ID: e19f2b431de24955b90ac23944d2d051
I20260808 17:02:20.772294 13833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:20.773442 13833 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:20.773608 13833 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6d63dd80a821452f96547ec8dbcdec36: Loaded TSK: 0
I20260808 17:02:20.773814 13833 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:02:20.840220 13818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:20.840411 13818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:20.840442 13818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:20.841634 13818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:20.841676 13818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:20.841711 13818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:20.843166 13818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.1.keytab
--principal=oryx/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:41065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=35007
--tserver_master_addrs=127.30.194.62:35419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:20.843391 13818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:20.843582 13818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:20.848186 13858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:20.848266 13863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.848354 13818 server_base.cc:1034] running on GCE node
W20260808 17:02:20.848201 13859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.848634 13818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:20.848860 13818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:20.850001 13818 hybrid_clock.cc:648] HybridClock initialized: now 1786208540849989 us; error 31 us; skew 500 ppm
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208540, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:20.852355 13818 init.cc:377] Logged in from keytab as oryx/127.30.194.1@KRBTEST.COM (short username oryx)
I20260808 17:02:20.852754 13818 webserver.cc:460] Webserver started at http://127.30.194.1:35007/ using document root <none> and password file <none>
I20260808 17:02:20.852967 13818 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:20.853029 13818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:20.854897 13818 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:20.855578 13872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.855792 13818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:20.855887 13818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "94c415fb180b42d0bd8954badb4730ef"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "16dbfa1ac939e560f2c3d6f6dddf1c4d"
server_key_iv: "9d0578793bf0b852a1fcfabf865b1a2c"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.856267 13818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:20.886541 13818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:20.888515 13818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:20.888625 13818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:20.888840 13818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:20.889175 13818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:20.889213 13818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.889243 13818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:20.889266 13818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.894770 13818 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:41065
I20260808 17:02:20.894825 14063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:41065 every 8 connection(s)
I20260808 17:02:20.895129 13818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:02:20.896415 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 13818
I20260808 17:02:20.896672 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:46205
--local_ip_for_outbound_sockets=127.30.194.2
--tserver_master_addrs=127.30.194.62:35419
--webserver_port=45535
--webserver_interface=127.30.194.2
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.194.2
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:20.903788 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:20.895760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:57985)
0808 17:02:20.895925 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:20.895970 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:20.896524 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:20.896988 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:20.896993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:20.897618 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:20.897815 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.897820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.899348 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.899350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.900122 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.900124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.900940 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.900940 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.901039 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.901471 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:20.901479 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:20.903658 (+ 2179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
W20260808 17:02:20.904022 14068 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:20.895847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:57985 (local address 127.30.194.62:35419)
0808 17:02:20.896063 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:20.896072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:20.896111 (+ 39us) server_negotiation.cc:407] Connection header received
0808 17:02:20.896743 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:20.896750 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:20.896807 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:20.896948 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:20.897948 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.899240 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.900221 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.900845 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.900961 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.903830 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:57985: BlockingRecv error: recv got EOF from 127.30.194.1:57985 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":94,"threads_started":1}
W20260808 17:02:20.904217 14064 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:35419 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.194.62:35419: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:20.908595 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:20.904435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:47563)
0808 17:02:20.904476 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:20.904484 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:20.904517 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:20.904646 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:20.904647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:20.904748 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:20.904790 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.904791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.905655 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.905655 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.906139 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.906141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.906573 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.906574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.906615 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.906929 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:20.906933 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:20.908554 (+ 1621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 17:02:20.908792 14068 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:20.904487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:47563 (local address 127.30.194.62:35419)
0808 17:02:20.904526 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:20.904528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:20.904538 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:20.904562 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:20.904563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:20.904576 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:20.904639 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:20.904852 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.905558 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.906218 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.906491 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.906579 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.908730 (+ 2151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:47563: BlockingRecv error: recv got EOF from 127.30.194.1:47563 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:20.913048 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:20.908809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:51631)
0808 17:02:20.908843 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:20.908852 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:20.908895 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:20.909134 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:20.909135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:20.909288 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:20.909321 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.909323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.909884 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.909885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.910680 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:20.910682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.911031 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.911032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:20.911084 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.911390 (+ 306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:20.911395 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:20.912983 (+ 1588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 17:02:20.913134 14072 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:20.908867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:51631 (local address 127.30.194.62:35419)
0808 17:02:20.908995 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:20.908996 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:20.909003 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:20.909022 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:20.909022 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:20.909030 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:20.909111 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:20.909397 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.909825 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.910762 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:20.910970 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:20.911017 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:20.913055 (+ 2038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:51631: BlockingRecv error: recv got EOF from 127.30.194.1:51631 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
W20260808 17:02:20.913197 14064 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:02:20.980582 14069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:20.980759 14069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:20.980780 14069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:20.981897 14069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:20.981937 14069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:20.981961 14069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:20.983320 14069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.2.keytab
--principal=oryx/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:46205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=45535
--tserver_master_addrs=127.30.194.62:35419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:20.983486 14069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:20.983645 14069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:20.988101 14080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.988271 14069 server_base.cc:1034] running on GCE node
W20260808 17:02:20.988134 14078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:20.988166 14083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:20.988580 14069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:20.988801 14069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:20.989938 14069 hybrid_clock.cc:648] HybridClock initialized: now 1786208540989923 us; error 28 us; skew 500 ppm
Aug 08 17:02:20 dist-test-slave-2j7r krb5kdc[12893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208540, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:20.992153 14069 init.cc:377] Logged in from keytab as oryx/127.30.194.2@KRBTEST.COM (short username oryx)
I20260808 17:02:20.992523 14069 webserver.cc:460] Webserver started at http://127.30.194.2:45535/ using document root <none> and password file <none>
I20260808 17:02:20.992714 14069 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:20.992775 14069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:20.993990 14069 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:02:20.994503 14091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.994656 14069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:20.994748 14069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "b6e54f9174d64eb9ad210b70abbf7830"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "7cac3a624d2fa36d954fe6412b90df0d"
server_key_iv: "2f162e3d0e59cef4e2c3a59b8cc79410"
server_key_version: "encryptionkey@0"
I20260808 17:02:20.995035 14069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:21.004437 14069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:21.006371 14069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:21.006498 14069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:21.006683 14069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:21.006947 14069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:21.006981 14069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:21.007005 14069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:21.007030 14069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:21.012403 14069 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:46205
I20260808 17:02:21.012444 14254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:46205 every 8 connection(s)
I20260808 17:02:21.012730 14069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:21.018890 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:21.013342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:56731)
0808 17:02:21.013482 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:21.013513 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:21.013916 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:21.014096 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:21.014100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:21.014596 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:21.014780 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.014785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.015367 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.015368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.016119 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.016121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.016396 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.016396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.016484 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.016836 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:21.016843 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:21.018765 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
W20260808 17:02:21.018900 14072 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:21.013370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:56731 (local address 127.30.194.62:35419)
0808 17:02:21.013446 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:21.013448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:21.013584 (+ 136us) server_negotiation.cc:407] Connection header received
0808 17:02:21.013998 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:21.013999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:21.014011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:21.014049 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:21.014842 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.015311 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.016177 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.016347 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.016387 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.018847 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:56731: BlockingRecv error: recv got EOF from 127.30.194.2:56731 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 17:02:21.019110 14255 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:35419 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.194.62:35419: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:21.022162 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 14069
I20260808 17:02:21.022424 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:34185
--local_ip_for_outbound_sockets=127.30.194.3
--tserver_master_addrs=127.30.194.62:35419
--webserver_port=34901
--webserver_interface=127.30.194.3
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.194.3
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:21.022809 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:21.019288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:40199)
0808 17:02:21.019381 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:21.019390 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:21.019430 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:21.019518 (+ 88us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:21.019519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:21.019615 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:21.019652 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.019653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.020172 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.020172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.020635 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.020636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.020896 (+ 260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.020896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.020938 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.021255 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:21.021259 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:21.022769 (+ 1510us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 17:02:21.022915 14072 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:21.019341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:40199 (local address 127.30.194.62:35419)
0808 17:02:21.019389 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:21.019391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:21.019419 (+ 28us) server_negotiation.cc:407] Connection header received
0808 17:02:21.019466 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:21.019467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:21.019476 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:21.019514 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:21.019695 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.020124 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.020686 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.020847 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.020891 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.022855 (+ 1964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:40199: BlockingRecv error: recv got EOF from 127.30.194.2:40199 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:21.026680 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:21.023039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:44897)
0808 17:02:21.023071 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:21.023079 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:21.023112 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:21.023226 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:21.023227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:21.023330 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:21.023352 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.023353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.023906 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.023907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.024363 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.024364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.024665 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.024666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.024713 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.025041 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:21.025044 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:21.026626 (+ 1582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 17:02:21.026741 14072 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:21.023077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:44897 (local address 127.30.194.62:35419)
0808 17:02:21.023107 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:21.023109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:21.023116 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:21.023154 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:21.023155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:21.023163 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:21.023214 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:21.023414 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.023843 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.024428 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.024611 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.024660 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.026697 (+ 2037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:44897: BlockingRecv error: recv got EOF from 127.30.194.2:44897 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 17:02:21.026791 14255 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:02:21.101931 14262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:21.102114 14262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:21.102144 14262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:21.103250 14262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:21.103291 14262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:21.103327 14262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:21.104696 14262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/krb5kdc/oryx_127.30.194.3.keytab
--principal=oryx/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:34185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=34901
--tserver_master_addrs=127.30.194.62:35419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:21.104905 14262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:21.105116 14262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:21.109676 14269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:21.109740 14275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:21.109840 14262 server_base.cc:1034] running on GCE node
W20260808 17:02:21.109912 14270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:21.110100 14262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:21.110323 14262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:21.111465 14262 hybrid_clock.cc:648] HybridClock initialized: now 1786208541111454 us; error 28 us; skew 500 ppm
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208541, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:21.113680 14262 init.cc:377] Logged in from keytab as oryx/127.30.194.3@KRBTEST.COM (short username oryx)
I20260808 17:02:21.114071 14262 webserver.cc:460] Webserver started at http://127.30.194.3:34901/ using document root <none> and password file <none>
I20260808 17:02:21.114234 14262 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:21.114280 14262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:21.115406 14262 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:21.115864 14283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:21.116027 14262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:21.116112 14262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "50a54411a9b24302bc089a8dacd65816"
format_stamp: "Formatted at 2026-08-08 17:02:20 on dist-test-slave-2j7r"
server_key: "45eb1ca19557723445819111198b61aa"
server_key_iv: "3cf8aa425c529d1181d8604fe4b0cf16"
server_key_version: "encryptionkey@0"
I20260808 17:02:21.116392 14262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:21.132503 14262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:21.134102 14262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:21.134202 14262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:21.134359 14262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:21.134599 14262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:21.134627 14262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:21.134651 14262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:21.134667 14262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:21.140064 14262 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:34185
I20260808 17:02:21.140105 14475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:34185 every 8 connection(s)
I20260808 17:02:21.140400 14262 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 17:02:21.146641 14072 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:21.141037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:35329 (local address 127.30.194.62:35419)
0808 17:02:21.141119 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:21.141121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:21.141279 (+ 158us) server_negotiation.cc:407] Connection header received
0808 17:02:21.141724 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:21.141725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:21.141735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:21.141775 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:21.142593 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.143059 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.143923 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.144091 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.144132 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.146586 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:35329: BlockingRecv error: recv got EOF from 127.30.194.3:35329 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:02:21.146670 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:21.141007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:35329)
0808 17:02:21.141174 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:21.141212 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:21.141638 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:21.141823 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:21.141826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:21.142348 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:21.142531 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.142535 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.143120 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.143120 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.143862 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.143864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.144141 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.144141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.144231 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.144577 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:21.144585 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:21.146497 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
W20260808 17:02:21.147023 14476 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:35419 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.194.62:35419: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
I20260808 17:02:21.147959 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 14262
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:21.151077 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:21.147227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:48871)
0808 17:02:21.147263 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:21.147273 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:21.147306 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:21.147485 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:21.147487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:21.147590 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:21.147631 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.147633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.148261 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.148262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.148776 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.148778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.149115 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.149116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.149170 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.149508 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:21.149512 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:21.151021 (+ 1509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 17:02:21.151150 14072 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:21.147357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:48871 (local address 127.30.194.62:35419)
0808 17:02:21.147404 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:21.147406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:21.147412 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:02:21.147427 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:21.147428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:21.147434 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:21.147476 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:21.147696 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.148187 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.148851 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.149030 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.149102 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.151097 (+ 1995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:48871: BlockingRecv error: recv got EOF from 127.30.194.3:48871 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:21.154735 14068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:21.148285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.30.194.62:35419)
0808 17:02:21.148320 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:21.148321 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:21.148479 (+ 158us) server_negotiation.cc:407] Connection header received
0808 17:02:21.148550 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:21.148551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:21.148562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:21.148608 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:21.149196 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.149748 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.150547 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.150726 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.150788 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.152071 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:21.152076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:21.152076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:21.152087 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:21.153505 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:21.153807 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:21.153810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:21.153811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:21.153905 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:21.154081 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:21.154082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:21.154082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:21.154308 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:21.154429 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:21.154548 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 17:02:21.154650 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:02:21.155097 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:21.151372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:42611)
0808 17:02:21.151425 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:21.151435 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:21.151472 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:21.151566 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:21.151567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:21.151682 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:21.151704 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.151705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.152244 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.152245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.152888 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.152891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.153252 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.153253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.153309 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.153613 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:21.153617 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:21.155044 (+ 1427us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 17:02:21.155159 14072 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:21.151441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:42611 (local address 127.30.194.62:35419)
0808 17:02:21.151486 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:21.151488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:21.151494 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:02:21.151512 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:21.151513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:21.151521 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:21.151564 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:21.151760 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.152180 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.152961 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.153188 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.153234 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.155118 (+ 1884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:42611: BlockingRecv error: recv got EOF from 127.30.194.3:42611 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 17:02:21.155299 14476 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:21 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:21.917636 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:21.913546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:36425)
0808 17:02:21.913662 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:21.913673 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:21.913724 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:21.913851 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:21.913851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:21.914003 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:21.914051 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.914052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.914632 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.914632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.915155 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:21.915156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.915470 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.915470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:21.915507 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.915913 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:21.915918 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:21.917571 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
W20260808 17:02:21.917742 14482 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:21.913623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:36425 (local address 127.30.194.62:35419)
0808 17:02:21.913741 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:21.913743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:21.913752 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:21.913772 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:21.913773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:21.913781 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:21.913842 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:21.914104 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.914583 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.915217 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:21.915416 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:21.915469 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:21.917654 (+ 2185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:36425: BlockingRecv error: recv got EOF from 127.30.194.1:36425 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
Aug 08 17:02:22 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:22 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:22.031543 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:22.027088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:34513)
0808 17:02:22.027242 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:22.027256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:22.027330 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:22.027475 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:22.027476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:22.027644 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:22.027696 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:22.027698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.028399 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.028400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:22.028940 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:22.028943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.029335 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.029336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:22.029393 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:22.029762 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:22.029767 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:22.031455 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":52,"threads_started":1}
W20260808 17:02:22.031615 14482 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:22.027189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:34513 (local address 127.30.194.62:35419)
0808 17:02:22.027267 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:22.027269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:22.027318 (+ 49us) server_negotiation.cc:407] Connection header received
0808 17:02:22.027389 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:22.027390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:22.027405 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:22.027466 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:22.027768 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.028316 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.029021 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.029269 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.029336 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:22.031556 (+ 2220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:34513: BlockingRecv error: recv got EOF from 127.30.194.2:34513 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 17:02:22 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:22 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:22.159600 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:22.155638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:52577)
0808 17:02:22.155751 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:22.155761 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:22.155814 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:22.155926 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:22.155926 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:22.156075 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:22.156123 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:22.156124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.156699 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.156700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:22.157299 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:22.157302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.157569 (+ 267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.157569 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:22.157604 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:22.157973 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:22.157976 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:22.159536 (+ 1560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":72,"threads_started":1}
W20260808 17:02:22.159730 14482 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:22.155699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:52577 (local address 127.30.194.62:35419)
0808 17:02:22.155757 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:22.155758 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:22.155802 (+ 44us) server_negotiation.cc:407] Connection header received
0808 17:02:22.155864 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:22.155865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:22.155885 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:22.155921 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:22.156168 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.156648 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.157353 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.157524 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.157564 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:22.159635 (+ 2071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:52577: BlockingRecv error: recv got EOF from 127.30.194.3:52577 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 17:02:22 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:22 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:22.922139 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:22.918039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:44367)
0808 17:02:22.918169 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:22.918181 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:22.918241 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:22.918359 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:22.918360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:22.918524 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:22.918571 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:22.918573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.919253 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.919255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:22.919798 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:22.919801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.920093 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.920094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:22.920132 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:22.920546 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:22.920550 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:22.922084 (+ 1534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
W20260808 17:02:22.922242 14487 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:22.918122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:44367 (local address 127.30.194.62:35419)
0808 17:02:22.918235 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:22.918237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:22.918245 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:22.918291 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:22.918292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:22.918301 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:22.918347 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:22.918623 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.919141 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.919862 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:22.920044 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:22.920083 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:22.922174 (+ 2091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:44367: BlockingRecv error: recv got EOF from 127.30.194.1:44367 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
Aug 08 17:02:23 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:23 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:23.036136 14489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:23.031993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:58125)
0808 17:02:23.032130 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:23.032140 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:23.032193 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:23.032329 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:23.032330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:23.032482 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:23.032531 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:23.032532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.033128 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.033128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:23.033667 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:23.033668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.033945 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.033945 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:23.033979 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:23.034349 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:23.034353 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:23.036064 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
W20260808 17:02:23.036216 14487 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:23.032015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:58125 (local address 127.30.194.62:35419)
0808 17:02:23.032063 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:23.032065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:23.032190 (+ 125us) server_negotiation.cc:407] Connection header received
0808 17:02:23.032263 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:23.032264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:23.032274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:23.032314 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:23.032574 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.033048 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.033721 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.033899 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.033942 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:23.036162 (+ 2220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:58125: BlockingRecv error: recv got EOF from 127.30.194.2:58125 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 17:02:23 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:23 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:23.164150 14490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:23.160101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:56459)
0808 17:02:23.160258 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:23.160268 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:23.160326 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:23.160437 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:23.160438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:23.160589 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:23.160637 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:23.160639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.161285 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.161286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:23.161813 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:23.161815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.162106 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.162107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:23.162141 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:23.162525 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:23.162529 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:23.164094 (+ 1565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
W20260808 17:02:23.164238 14487 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:23.160209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:56459 (local address 127.30.194.62:35419)
0808 17:02:23.160265 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:23.160267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:23.160319 (+ 52us) server_negotiation.cc:407] Connection header received
0808 17:02:23.160366 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:23.160367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:23.160380 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:23.160425 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:23.160682 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.161212 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.161868 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.162053 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.162098 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:23.164179 (+ 2081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:56459: BlockingRecv error: recv got EOF from 127.30.194.3:56459 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 17:02:23 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:23 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:23.926910 14491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:23.922612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:51541)
0808 17:02:23.922750 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:23.922760 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:23.922813 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:23.922949 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:23.922949 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:23.923113 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:23.923162 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:23.923164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.923825 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.923825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:23.924332 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:23.924333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.924672 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.924672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:23.924709 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:23.925144 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:23.925150 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:23.926844 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
W20260808 17:02:23.927027 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:23.922704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:51541 (local address 127.30.194.62:35419)
0808 17:02:23.922832 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:23.922834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:23.922842 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:23.922863 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:23.922864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:23.922874 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:23.922937 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:23.923221 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.923759 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.924405 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:23.924610 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:23.924672 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:23.926946 (+ 2274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:51541: BlockingRecv error: recv got EOF from 127.30.194.1:51541 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
Aug 08 17:02:24 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:24 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:24.041065 14494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:24.036566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:45099)
0808 17:02:24.036705 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:24.036716 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:24.036784 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:24.036918 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:24.036919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:24.037109 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:24.037166 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:24.037169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.037803 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.037804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:24.038351 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:24.038354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.038832 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.038833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:24.038907 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:24.039325 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:24.039330 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:24.041006 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
W20260808 17:02:24.041196 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:24.036631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:45099 (local address 127.30.194.62:35419)
0808 17:02:24.036692 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:24.036694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:24.036772 (+ 78us) server_negotiation.cc:407] Connection header received
0808 17:02:24.036847 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:24.036848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:24.036860 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:24.036900 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:24.037238 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.037730 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.038454 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.038754 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.038829 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:24.041128 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:45099: BlockingRecv error: recv got EOF from 127.30.194.2:45099 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 17:02:24 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:24 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:24.168908 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:24.164569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:54419)
0808 17:02:24.164704 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:24.164715 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:24.164787 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:24.164919 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:24.164920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:24.165142 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:24.165198 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:24.165200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.165945 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.165946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:24.166477 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:24.166479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.166800 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.166801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:24.166854 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:24.167249 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:24.167254 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:24.168835 (+ 1581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
W20260808 17:02:24.168982 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:24.164642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:54419 (local address 127.30.194.62:35419)
0808 17:02:24.164703 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:24.164705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:24.164767 (+ 62us) server_negotiation.cc:407] Connection header received
0808 17:02:24.164845 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:24.164846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:24.164860 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:24.164899 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:24.165299 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.165862 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.166551 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.166740 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.166794 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:24.168928 (+ 2134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:54419: BlockingRecv error: recv got EOF from 127.30.194.3:54419 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 17:02:24 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:24 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:24.934864 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:24.927316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:50699)
0808 17:02:24.927449 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:24.927459 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:24.927503 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:24.927622 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:24.927624 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:24.927778 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:24.927827 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:24.927829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.928431 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.928432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:24.928952 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:24.928954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.929285 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.929286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:24.929325 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:24.933186 (+ 3861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:24.933192 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:24.934789 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
W20260808 17:02:24.934954 14497 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:24.927400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:50699 (local address 127.30.194.62:35419)
0808 17:02:24.927511 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:24.927513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:24.927520 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:24.927540 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:24.927541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:24.927550 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:24.927608 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:24.927877 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.928374 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.929006 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:24.929235 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:24.929287 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:24.934892 (+ 5605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:50699: BlockingRecv error: recv got EOF from 127.30.194.1:50699 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
Aug 08 17:02:25 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:25 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:25.045598 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:25.041468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:47363)
0808 17:02:25.041596 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:25.041608 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:25.041677 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:25.041819 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:25.041820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:25.041987 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:25.042036 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:25.042038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.042627 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.042628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:25.043154 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:25.043156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.043443 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.043443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:25.043477 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:25.043854 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:25.043859 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:25.045541 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
W20260808 17:02:25.045681 14497 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:25.041532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:47363 (local address 127.30.194.62:35419)
0808 17:02:25.041591 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:25.041594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:25.041664 (+ 70us) server_negotiation.cc:407] Connection header received
0808 17:02:25.041750 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:25.041751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:25.041763 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:25.041809 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:25.042085 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.042575 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.043214 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.043394 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.043440 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:25.045624 (+ 2184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:47363: BlockingRecv error: recv got EOF from 127.30.194.2:47363 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 17:02:25 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:25 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:25.173746 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:25.169388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:54033)
0808 17:02:25.169503 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:25.169513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:25.169571 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:25.169692 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:25.169693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:25.169848 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:25.169895 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:25.169897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.170523 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.170524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:25.171101 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:25.171104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.171397 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.171398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:25.171438 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:25.171918 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:25.171922 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:25.173690 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
W20260808 17:02:25.173836 14497 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:25.169442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:54033 (local address 127.30.194.62:35419)
0808 17:02:25.169519 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:25.169521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:25.169556 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:02:25.169624 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:25.169625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:25.169638 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:25.169700 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:25.169972 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.170458 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.171167 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.171346 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.171394 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:25.173779 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:54033: BlockingRecv error: recv got EOF from 127.30.194.3:54033 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 17:02:25 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:25 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:25.940704 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:25.935354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:39983)
0808 17:02:25.935491 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:25.935501 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:25.935568 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:25.935782 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:25.935783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:25.935984 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:25.936046 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:25.936049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.937004 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.937005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:25.937574 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:25.937576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.938045 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.938046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:25.938083 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:25.938533 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:25.938538 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:25.940601 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":56,"threads_started":1}
W20260808 17:02:25.940773 14502 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:25.935420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:39983 (local address 127.30.194.62:35419)
0808 17:02:25.935551 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:25.935554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:25.935565 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:02:25.935632 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:25.935633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:25.935645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:25.935723 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:25.936113 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.936927 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.937652 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:25.937978 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:25.938043 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:25.940697 (+ 2654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:39983: BlockingRecv error: recv got EOF from 127.30.194.1:39983 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
Aug 08 17:02:26 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:26 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:26.050171 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:26.046005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:33325)
0808 17:02:26.046137 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:26.046150 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:26.046206 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:26.046345 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:26.046346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:26.046494 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:26.046565 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:26.046567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.047161 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.047161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:26.047668 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:26.047670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.047937 (+ 267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.047937 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:26.047980 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:26.048384 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:26.048389 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:26.050102 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
W20260808 17:02:26.050238 14502 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:26.046129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:33325 (local address 127.30.194.62:35419)
0808 17:02:26.046185 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:26.046187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:26.046196 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:26.046276 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:26.046277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:26.046287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:26.046334 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:26.046613 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.047110 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.047719 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.047893 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.047934 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:26.050186 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:33325: BlockingRecv error: recv got EOF from 127.30.194.2:33325 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 17:02:26 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:26 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:26.178316 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:26.174156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:33457)
0808 17:02:26.174279 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:26.174289 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:26.174352 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:26.174464 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:26.174465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:26.174618 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:26.174666 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:26.174667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.175246 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.175247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:26.175760 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:26.175761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.176061 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.176061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:26.176102 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:26.176537 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:26.176542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:26.178260 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
W20260808 17:02:26.178407 14502 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:26.174239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:33457 (local address 127.30.194.62:35419)
0808 17:02:26.174283 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:26.174285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:26.174357 (+ 72us) server_negotiation.cc:407] Connection header received
0808 17:02:26.174404 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:26.174405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:26.174416 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:26.174452 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:26.174713 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.175194 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.175820 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.176001 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.176045 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:26.178343 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:33457: BlockingRecv error: recv got EOF from 127.30.194.3:33457 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 17:02:26 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:26 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:26.945601 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:26.941317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:50259)
0808 17:02:26.941440 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:26.941450 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:26.941499 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:26.941615 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:26.941616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:26.941767 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:26.941815 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:26.941817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.942455 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.942456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:26.942989 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:26.942991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.943296 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.943297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:26.943338 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:26.943821 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:26.943827 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:26.945544 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
W20260808 17:02:26.945704 14507 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:26.941371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:50259 (local address 127.30.194.62:35419)
0808 17:02:26.941495 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:26.941497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:26.941507 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:26.941547 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:26.941547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:26.941557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:26.941606 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:26.941869 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.942396 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.943057 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:26.943246 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:26.943292 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:26.945635 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:50259: BlockingRecv error: recv got EOF from 127.30.194.1:50259 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
Aug 08 17:02:27 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:27 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:27.054687 14508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:27.050572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:33645)
0808 17:02:27.050692 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:27.050701 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:27.050755 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:27.050866 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:27.050866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:27.051030 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:27.051080 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:27.051082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.051655 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.051656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:27.052194 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:27.052195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.052466 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.052467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:27.052501 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:27.052880 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:27.052884 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:27.054618 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
W20260808 17:02:27.054813 14507 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:27.050616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:33645 (local address 127.30.194.62:35419)
0808 17:02:27.050659 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:27.050661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:27.050751 (+ 90us) server_negotiation.cc:407] Connection header received
0808 17:02:27.050797 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:27.050798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:27.050809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:27.050855 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:27.051129 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.051604 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.052247 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.052420 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.052463 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:27.054744 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:33645: BlockingRecv error: recv got EOF from 127.30.194.2:33645 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 17:02:27 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:27 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:27.182928 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:27.178746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:50371)
0808 17:02:27.178879 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:27.178889 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:27.178937 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:27.179060 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:27.179061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:27.179225 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:27.179276 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:27.179278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.179864 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.179864 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:27.180384 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:27.180385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.180648 (+ 263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.180648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:27.180684 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:27.181075 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:27.181079 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:27.182868 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
W20260808 17:02:27.183118 14507 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:27.178854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:50371 (local address 127.30.194.62:35419)
0808 17:02:27.178909 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:27.178911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:27.178922 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:02:27.178995 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:27.178996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:27.179007 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:27.179048 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:27.179328 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.179811 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.180436 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.180602 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.180644 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:27.183045 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:50371: BlockingRecv error: recv got EOF from 127.30.194.3:50371 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 17:02:27 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:27 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:27.950335 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:27.946012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:60805)
0808 17:02:27.946152 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:27.946162 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:27.946216 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:27.946342 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:27.946342 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:27.946497 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:27.946565 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:27.946566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.947155 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.947155 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:27.947688 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:27.947689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.948008 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.948009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:27.948045 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:27.948466 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:27.948470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:27.950262 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
W20260808 17:02:27.950387 14512 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:27.946071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:60805 (local address 127.30.194.62:35419)
0808 17:02:27.946182 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:27.946184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:27.946194 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:02:27.946263 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:27.946264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:27.946274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:27.946333 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:27.946614 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.947102 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.947751 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:27.947954 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:27.948006 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:27.950331 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:60805: BlockingRecv error: recv got EOF from 127.30.194.1:60805 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
Aug 08 17:02:28 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:28 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:28.059401 14513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:28.055080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:52357)
0808 17:02:28.055231 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:28.055242 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:28.055319 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:28.055481 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:28.055482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:28.055704 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:28.055762 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:28.055765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.056377 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.056378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:28.056920 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:28.056922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.057242 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.057244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:28.057280 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:28.057666 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:28.057671 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:28.059326 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
W20260808 17:02:28.059466 14512 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:28.055168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:52357 (local address 127.30.194.62:35419)
0808 17:02:28.055219 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:28.055221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:28.055309 (+ 88us) server_negotiation.cc:407] Connection header received
0808 17:02:28.055390 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:28.055391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:28.055403 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:28.055467 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:28.055828 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.056309 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.056979 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.057191 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.057234 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:28.059418 (+ 2184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:52357: BlockingRecv error: recv got EOF from 127.30.194.2:52357 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 17:02:28 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:28 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:28.187633 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:28.183350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:35247)
0808 17:02:28.183486 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:28.183497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:28.183565 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:28.183696 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:28.183697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:28.183874 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:28.183920 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:28.183922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.184547 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.184549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:28.185072 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:28.185074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.185411 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.185412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:28.185468 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:28.185877 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:28.185881 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:28.187545 (+ 1664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
W20260808 17:02:28.187692 14512 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:28.183407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:35247 (local address 127.30.194.62:35419)
0808 17:02:28.183469 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:28.183471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:28.183556 (+ 85us) server_negotiation.cc:407] Connection header received
0808 17:02:28.183627 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:28.183628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:28.183642 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:28.183682 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:28.183993 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.184477 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.185171 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.185351 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.185406 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:28.187638 (+ 2232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:35247: BlockingRecv error: recv got EOF from 127.30.194.3:35247 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 17:02:28 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:28 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:28.955086 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:28.950804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:47653)
0808 17:02:28.950932 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:28.950943 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:28.950992 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:28.951158 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:28.951159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:28.951310 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:28.951360 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:28.951362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.951975 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.951976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:28.952491 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:28.952492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.952799 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.952800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:28.952836 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:28.953277 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:28.953281 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:28.955009 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":57,"threads_started":1}
W20260808 17:02:28.955171 14518 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:28.950929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:47653 (local address 127.30.194.62:35419)
0808 17:02:28.951049 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:28.951051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:28.951058 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:28.951079 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:28.951080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:28.951089 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:28.951148 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:28.951422 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.951922 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.952547 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:28.952748 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:28.952797 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:28.955109 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:47653: BlockingRecv error: recv got EOF from 127.30.194.1:47653 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
Aug 08 17:02:29 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:29 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:29.064463 14520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:29.059881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:43485)
0808 17:02:29.060033 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:29.060044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:29.060105 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:29.060211 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:29.060212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:29.060367 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:29.060444 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:29.060446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.061255 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.061255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:29.061777 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:29.061778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.062092 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.062093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:29.062146 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:29.062631 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:29.062637 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:29.064386 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
W20260808 17:02:29.064530 14518 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:29.059963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:43485 (local address 127.30.194.62:35419)
0808 17:02:29.060025 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:29.060027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:29.060089 (+ 62us) server_negotiation.cc:407] Connection header received
0808 17:02:29.060145 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:29.060146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:29.060157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:29.060202 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:29.060528 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.061198 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.061837 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.062026 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.062088 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:29.064477 (+ 2389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:43485: BlockingRecv error: recv got EOF from 127.30.194.2:43485 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 17:02:29 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:29 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:29.192188 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:29.188075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:34739)
0808 17:02:29.188195 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:29.188205 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:29.188262 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:29.188375 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:29.188376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:29.188531 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:29.188580 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:29.188582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.189254 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.189255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:29.189824 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:29.189826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.190127 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.190128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:29.190165 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:29.190546 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:29.190549 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:29.192114 (+ 1565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
W20260808 17:02:29.192276 14518 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:29.188127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:34739 (local address 127.30.194.62:35419)
0808 17:02:29.188171 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:29.188172 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:29.188258 (+ 86us) server_negotiation.cc:407] Connection header received
0808 17:02:29.188314 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:29.188315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:29.188327 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:29.188364 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:29.188639 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.189187 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.189890 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.190074 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.190125 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:29.192221 (+ 2096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:34739: BlockingRecv error: recv got EOF from 127.30.194.3:34739 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 17:02:29 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:29 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:29.964018 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:29.955527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:54827)
0808 17:02:29.955641 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:29.955652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:29.955703 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:29.955830 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:29.955831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:29.955987 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:29.956035 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:29.956036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.956622 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.956622 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:29.957201 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:29.957204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.957504 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.957505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:29.957552 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:29.962152 (+ 4600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:29.962157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:29.963945 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":64,"threads_started":1}
W20260808 17:02:29.964090 14525 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:29.955580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:54827 (local address 127.30.194.62:35419)
0808 17:02:29.955697 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:29.955699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:29.955707 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:29.955756 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:29.955757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:29.955767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:29.955818 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:29.956081 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.956569 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.957261 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:29.957442 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:29.957483 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:29.964030 (+ 6547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:54827: BlockingRecv error: recv got EOF from 127.30.194.1:54827 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
Aug 08 17:02:30 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:30 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:30.069419 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:30.065058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:38089)
0808 17:02:30.065201 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:30.065212 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:30.065278 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:30.065391 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:30.065391 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:30.065546 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:30.065594 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:30.065595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.066230 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.066232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:30.066790 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:30.066791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.067074 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.067075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:30.067109 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:30.067491 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:30.067496 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:30.069340 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
W20260808 17:02:30.069511 14525 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:30.065184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:38089 (local address 127.30.194.62:35419)
0808 17:02:30.065226 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:30.065228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:30.065261 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:02:30.065323 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:30.065324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:30.065335 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:30.065381 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:30.065641 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.066164 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.066849 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.067029 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.067070 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:30.069426 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:38089: BlockingRecv error: recv got EOF from 127.30.194.2:38089 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 17:02:30 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:30 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:30.196913 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:30.192620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:60331)
0808 17:02:30.192791 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:30.192803 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:30.192863 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:30.192964 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:30.192965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:30.193155 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:30.193205 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:30.193208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.193819 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.193820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:30.194373 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:30.194374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.194678 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.194680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:30.194740 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:30.195141 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:30.195145 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:30.196844 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
W20260808 17:02:30.196980 14525 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:30.192688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:60331 (local address 127.30.194.62:35419)
0808 17:02:30.192760 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:30.192762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:30.192857 (+ 95us) server_negotiation.cc:407] Connection header received
0808 17:02:30.192903 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:30.192904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:30.192916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:30.192960 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:30.193277 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.193759 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.194430 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.194602 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.194646 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:30.196927 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:60331: BlockingRecv error: recv got EOF from 127.30.194.3:60331 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 17:02:30 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:30 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:30.968514 14529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:30.964431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:54383)
0808 17:02:30.964543 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:30.964553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:30.964603 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:30.964730 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:30.964731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:30.964883 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:30.964931 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:30.964932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.965622 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.965623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:30.966160 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:30.966162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.966456 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.966456 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:30.966495 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:30.966868 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:30.966872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:30.968458 (+ 1586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":66,"threads_started":1}
W20260808 17:02:30.968614 14530 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:30.964494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:54383 (local address 127.30.194.62:35419)
0808 17:02:30.964605 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:30.964607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:30.964618 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:02:30.964642 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:30.964643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:30.964653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:30.964709 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:30.964980 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.965559 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.966220 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:30.966404 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:30.966452 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:30.968546 (+ 2094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:54383: BlockingRecv error: recv got EOF from 127.30.194.1:54383 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
Aug 08 17:02:31 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:31 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:31.074491 14531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:31.069878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:34517)
0808 17:02:31.070004 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:31.070015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:31.070082 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:31.070215 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:31.070216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:31.070411 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:31.070457 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:31.070459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.071118 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.071119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:31.071752 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:31.071754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.072102 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.072103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:31.072157 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:31.072588 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:31.072594 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:31.074431 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
W20260808 17:02:31.074589 14530 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:31.069922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:34517 (local address 127.30.194.62:35419)
0808 17:02:31.069985 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:31.069988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:31.070071 (+ 83us) server_negotiation.cc:407] Connection header received
0808 17:02:31.070144 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:31.070145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:31.070157 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:31.070204 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:31.070528 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.071043 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.071832 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.072039 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.072096 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:31.074529 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:34517: BlockingRecv error: recv got EOF from 127.30.194.2:34517 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 17:02:31 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:31 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:31.201625 14532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:31.197354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:47577)
0808 17:02:31.197476 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:31.197489 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:31.197550 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:31.197663 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:31.197664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:31.197839 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:31.197884 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:31.197886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.198535 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.198536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:31.199057 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:31.199058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.199384 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.199385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:31.199437 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:31.199820 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:31.199824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:31.201567 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
W20260808 17:02:31.201766 14530 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:31.197498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:47577 (local address 127.30.194.62:35419)
0808 17:02:31.197551 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:31.197553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:31.197561 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:02:31.197600 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:31.197601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:31.197614 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:31.197657 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:31.197962 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.198471 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.199128 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.199324 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.199378 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:31.201694 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:47577: BlockingRecv error: recv got EOF from 127.30.194.3:47577 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 17:02:31 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:31 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:31.973518 14533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:31.968939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:39691)
0808 17:02:31.969065 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:31.969075 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:31.969166 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:31.969292 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:31.969293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:31.969460 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:31.969508 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:31.969509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.970206 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.970208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:31.970875 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:31.970879 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.971220 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.971222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:31.971282 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:31.971723 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:31.971729 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:31.973430 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
W20260808 17:02:31.973589 14534 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:31.969019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:39691 (local address 127.30.194.62:35419)
0808 17:02:31.969132 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:31.969134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:31.969147 (+ 13us) server_negotiation.cc:407] Connection header received
0808 17:02:31.969215 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:31.969215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:31.969226 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:31.969281 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:31.969555 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.970132 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.970943 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:31.971143 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:31.971204 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:31.973526 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:39691: BlockingRecv error: recv got EOF from 127.30.194.1:39691 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":49,"threads_started":1}
Aug 08 17:02:32 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:32 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:32.079314 14535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:32.074952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:37497)
0808 17:02:32.075089 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:32.075099 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:32.075157 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:32.075283 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:32.075284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:32.075452 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:32.075512 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:32.075514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.076292 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.076293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:32.076835 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:32.076837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.077136 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.077137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:32.077173 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:32.077562 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:32.077566 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:32.079223 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
W20260808 17:02:32.079382 14534 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:32.075025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:37497 (local address 127.30.194.62:35419)
0808 17:02:32.075081 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:32.075083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:32.075156 (+ 73us) server_negotiation.cc:407] Connection header received
0808 17:02:32.075214 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:32.075214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:32.075226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:32.075263 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:32.075583 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.076222 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.076897 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.077079 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.077136 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:32.079318 (+ 2182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:37497: BlockingRecv error: recv got EOF from 127.30.194.2:37497 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 17:02:32 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:32 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:32.206429 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:32.202059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:46835)
0808 17:02:32.202230 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:32.202242 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:32.202315 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:32.202448 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:32.202449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:32.202621 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:32.202666 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:32.202668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.203299 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.203300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:32.203851 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:32.203853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.204163 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.204164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:32.204219 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:32.204617 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:32.204621 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:32.206358 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":82,"threads_started":1}
W20260808 17:02:32.206516 14534 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:32.202148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:46835 (local address 127.30.194.62:35419)
0808 17:02:32.202228 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:32.202230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:32.202302 (+ 72us) server_negotiation.cc:407] Connection header received
0808 17:02:32.202371 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:32.202372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:32.202388 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:32.202433 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:32.202730 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.203228 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.203922 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.204106 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.204158 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:32.206455 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:46835: BlockingRecv error: recv got EOF from 127.30.194.3:46835 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 17:02:32 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:32 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:32.978348 14538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:32.973931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:57779)
0808 17:02:32.974063 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:32.974074 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:32.974130 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:32.974277 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:32.974277 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:32.974453 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:32.974501 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:32.974503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.975170 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.975171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:32.975712 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:32.975714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.976038 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.976038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:32.976076 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:32.976478 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:32.976483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:32.978281 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
W20260808 17:02:32.978427 14539 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:32.973999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:57779 (local address 127.30.194.62:35419)
0808 17:02:32.974112 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:32.974114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:32.974123 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:32.974181 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:32.974181 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:32.974192 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:32.974268 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:32.974588 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.975106 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.975780 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:32.975986 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:32.976036 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:32.978369 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:57779: BlockingRecv error: recv got EOF from 127.30.194.1:57779 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":50,"threads_started":1}
Aug 08 17:02:33 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:33 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:33.084108 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:33.079738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:48117)
0808 17:02:33.079882 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:33.079893 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:33.079963 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:33.080088 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:33.080089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:33.080273 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:33.080322 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:33.080323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.080926 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.080927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:33.081523 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:33.081526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.081836 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.081838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:33.081903 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:33.082282 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:33.082286 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:33.084042 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":60,"threads_started":1}
W20260808 17:02:33.084185 14539 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:33.079842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:48117 (local address 127.30.194.62:35419)
0808 17:02:33.079900 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:33.079902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:33.079956 (+ 54us) server_negotiation.cc:407] Connection header received
0808 17:02:33.080011 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:33.080012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:33.080023 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:33.080068 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:33.080385 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.080864 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.081586 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.081763 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.081805 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:33.084128 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:48117: BlockingRecv error: recv got EOF from 127.30.194.2:48117 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 17:02:33 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:33 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:33.211452 14541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:33.206908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:58389)
0808 17:02:33.207061 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:33.207072 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:33.207139 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:33.207267 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:33.207268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:33.207450 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:33.207492 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:33.207493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.208131 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.208132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:33.208666 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:33.208668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.208995 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.208996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:33.209057 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:33.209492 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:33.209497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:33.211376 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
W20260808 17:02:33.211517 14539 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:33.206988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:58389 (local address 127.30.194.62:35419)
0808 17:02:33.207058 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:33.207060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:33.207131 (+ 71us) server_negotiation.cc:407] Connection header received
0808 17:02:33.207190 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:33.207191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:33.207205 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:33.207254 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:33.207567 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.208057 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.208745 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.208933 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.208987 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:33.211466 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:58389: BlockingRecv error: recv got EOF from 127.30.194.3:58389 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 17:02:33 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:33 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:33.983385 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:33.978882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:38809)
0808 17:02:33.979038 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:33.979049 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:33.979114 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:33.979243 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:33.979244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:33.979422 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:33.979473 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:33.979475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.980102 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.980104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:33.980633 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:33.980635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.980953 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.980955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:33.981010 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:33.981461 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:33.981467 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:33.983307 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
W20260808 17:02:33.983480 14544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:33.978932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:38809 (local address 127.30.194.62:35419)
0808 17:02:33.979046 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:33.979048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:33.979106 (+ 58us) server_negotiation.cc:407] Connection header received
0808 17:02:33.979158 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:33.979159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:33.979169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:33.979227 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:33.979528 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.980013 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.980695 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:33.980879 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:33.980920 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:33.983418 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:38809: BlockingRecv error: recv got EOF from 127.30.194.1:38809 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
Aug 08 17:02:34 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:34 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:34.088796 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:34.084534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:53111)
0808 17:02:34.084662 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:34.084672 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:34.084736 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:34.084849 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:34.084850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:34.085026 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:34.085072 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:34.085074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.085692 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.085693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:34.086231 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:34.086233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.086512 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.086513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:34.086547 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:34.086937 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:34.086942 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:34.088717 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
W20260808 17:02:34.088866 14544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:34.084621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:53111 (local address 127.30.194.62:35419)
0808 17:02:34.084665 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:34.084667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:34.084723 (+ 56us) server_negotiation.cc:407] Connection header received
0808 17:02:34.084790 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:34.084791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:34.084800 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:34.084839 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:34.085137 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.085636 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.086290 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.086468 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.086508 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:34.088813 (+ 2305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:53111: BlockingRecv error: recv got EOF from 127.30.194.2:53111 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 17:02:34 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:34 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:34.216236 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:34.211924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:57119)
0808 17:02:34.212081 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:34.212092 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:34.212151 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:34.212300 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:34.212301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:34.212474 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:34.212525 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:34.212526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.213167 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.213167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:34.213707 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:34.213708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.214001 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.214001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:34.214042 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:34.214446 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:34.214451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:34.216157 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
W20260808 17:02:34.216307 14544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:34.211972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:57119 (local address 127.30.194.62:35419)
0808 17:02:34.212016 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:34.212018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:34.212146 (+ 128us) server_negotiation.cc:407] Connection header received
0808 17:02:34.212204 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:34.212205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:34.212219 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:34.212256 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:34.212576 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.213062 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.213770 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.213951 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.213992 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:34.216252 (+ 2260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:57119: BlockingRecv error: recv got EOF from 127.30.194.3:57119 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 17:02:34 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:34 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:34.991730 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:34.983857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:43085)
0808 17:02:34.984015 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:34.984026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:34.984080 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:34.984235 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:34.984236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:34.984409 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:34.984459 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:34.984460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.985129 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.985130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:34.985657 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:34.985659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.986022 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.986023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:34.986060 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:34.989889 (+ 3829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:34.989894 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:34.991671 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
W20260808 17:02:34.991860 14548 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:34.983958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:43085 (local address 127.30.194.62:35419)
0808 17:02:34.984088 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:34.984090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:34.984099 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:34.984121 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:34.984121 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:34.984132 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:34.984228 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:34.984518 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.985036 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.985739 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:34.985957 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:34.986021 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:34.991776 (+ 5755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:43085: BlockingRecv error: recv got EOF from 127.30.194.1:43085 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
Aug 08 17:02:35 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:35 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:35.093493 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:35.089227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:47935)
0808 17:02:35.089346 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:35.089357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:35.089424 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:35.089541 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:35.089541 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:35.089700 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:35.089747 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:35.089749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.090414 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.090415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:35.090925 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:35.090926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.091203 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.091204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:35.091238 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:35.091652 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:35.091657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:35.093426 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
W20260808 17:02:35.093544 14548 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:35.089300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:47935 (local address 127.30.194.62:35419)
0808 17:02:35.089372 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:35.089374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:35.089411 (+ 37us) server_negotiation.cc:407] Connection header received
0808 17:02:35.089480 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:35.089481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:35.089491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:35.089531 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:35.089832 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.090354 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.090980 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.091159 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.091201 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:35.093496 (+ 2295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:47935: BlockingRecv error: recv got EOF from 127.30.194.2:47935 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 17:02:35 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:35 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.3@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:35.221122 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:35.216696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.3:33639)
0808 17:02:35.216838 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:35.216848 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:35.216915 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:35.217027 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:35.217028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:35.217205 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:35.217251 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:35.217252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.217868 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.217878 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:35.218424 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:35.218426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.218767 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.218768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:35.218812 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:35.219216 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:35.219220 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:35.221037 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
W20260808 17:02:35.221212 14548 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:35.216760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:33639 (local address 127.30.194.62:35419)
0808 17:02:35.216841 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:35.216843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:35.216902 (+ 59us) server_negotiation.cc:407] Connection header received
0808 17:02:35.216954 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:35.216955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:35.216968 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:35.217014 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:35.217321 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.217811 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.218513 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.218702 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.218765 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:35.221150 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.3:33639: BlockingRecv error: recv got EOF from 127.30.194.3:33639 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 17:02:35 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:35 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.1@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:35.996477 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:35.992180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.1:51347)
0808 17:02:35.992295 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:35.992305 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:35.992354 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:35.992490 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:35.992490 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:35.992649 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:35.992703 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:35.992704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.993392 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.993392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:35.993926 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:35.993928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.994232 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.994232 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:35.994268 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:35.994656 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:35.994660 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:35.996383 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
W20260808 17:02:35.996554 14555 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:35.992278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:51347 (local address 127.30.194.62:35419)
0808 17:02:35.992383 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:35.992385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:35.992392 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:02:35.992412 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:35.992412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:35.992422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:35.992480 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:35.992795 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.993328 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.993991 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:35.994176 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:35.994233 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:35.996488 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.1:51347: BlockingRecv error: recv got EOF from 127.30.194.1:51347 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":47,"threads_started":1}
Aug 08 17:02:36 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:02:36 dist-test-slave-2j7r krb5kdc[12893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.194.2@KRBTEST.COM for oryx/127.30.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:02:36.098151 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:36.093950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35419 (local address 127.30.194.2:49415)
0808 17:02:36.094070 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:36.094080 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:36.094137 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:36.094272 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:36.094273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:36.094436 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:36.094485 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:36.094487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.095082 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.095083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:36.095605 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:36.095606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.095907 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.095907 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:36.095942 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:36.096389 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:36.096393 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:36.098063 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.194.62:35419: Server oryx/127.30.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
W20260808 17:02:36.098238 14555 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:36.094005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:49415 (local address 127.30.194.62:35419)
0808 17:02:36.094088 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:36.094089 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:36.094124 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:02:36.094188 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:36.094189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:36.094200 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:36.094261 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:36.094536 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.095031 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.095668 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.095857 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.095904 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:36.098157 (+ 2253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.194.2:49415: BlockingRecv error: recv got EOF from 127.30.194.2:49415 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 17:02:36.155230 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 13818
I20260808 17:02:36.173914 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 14069
I20260808 17:02:36.183914 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 14262
I20260808 17:02:36.189939 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 13721
2026-08-08T17:02:36Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17528 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T17:02:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:36Z Disabled control of system clock
I20260808 17:02:36.220955 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:42277
--webserver_interface=127.30.194.62
--webserver_port=0
--builtin_ntp_servers=127.30.194.20:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:42277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 17:02:36.302084 14565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:36.302263 14565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:36.302282 14565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:36.303345 14565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:36.303376 14565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:36.303388 14565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:36.303399 14565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:36.303412 14565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:36.304693 14565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:42277
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:42277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:36.304879 14565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:36.305107 14565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:36.309778 14571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:36.309814 14565 server_base.cc:1034] running on GCE node
W20260808 17:02:36.309814 14572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:36.310030 14574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:36.310339 14565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:36.310603 14565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:36.311756 14565 hybrid_clock.cc:648] HybridClock initialized: now 1786208556311740 us; error 29 us; skew 500 ppm
I20260808 17:02:36.312917 14565 webserver.cc:460] Webserver started at http://127.30.194.62:41653/ using document root <none> and password file <none>
I20260808 17:02:36.313160 14565 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:36.313211 14565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:36.313345 14565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:36.314342 14565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "4d9ffec90c144f738640e6b0d7319b4c"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "6d1d8120ea8b1ff8fb47712c90a4ce94"
server_key_iv: "2d1ae67859595ebdc829dae32a4b2c2c"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.314678 14565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "4d9ffec90c144f738640e6b0d7319b4c"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "6d1d8120ea8b1ff8fb47712c90a4ce94"
server_key_iv: "2d1ae67859595ebdc829dae32a4b2c2c"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.315845 14565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:02:36.316491 14582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.316638 14565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:36.316722 14565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "4d9ffec90c144f738640e6b0d7319b4c"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "6d1d8120ea8b1ff8fb47712c90a4ce94"
server_key_iv: "2d1ae67859595ebdc829dae32a4b2c2c"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.316787 14565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:36.328454 14565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:36.328681 14565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:36.328801 14565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:36.332393 14565 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:42277
I20260808 17:02:36.332425 14670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:42277 every 8 connection(s)
I20260808 17:02:36.332698 14565 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:36.333331 14671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:36.334898 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 14565
I20260808 17:02:36.334983 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:36.335150 31496 external_mini_cluster.cc:1428] Setting key 4737ab0ac0a135d2d16d5b06ba8ee4be
I20260808 17:02:36.335974 14671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c: Bootstrap starting.
I20260808 17:02:36.336655 14671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:36.336987 14671 log.cc:824] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:36.337746 14671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c: No bootstrap required, opened a new log
I20260808 17:02:36.339727 14671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9ffec90c144f738640e6b0d7319b4c" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42277 } }
I20260808 17:02:36.339854 14671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:36.339883 14671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9ffec90c144f738640e6b0d7319b4c, State: Initialized, Role: FOLLOWER
I20260808 17:02:36.340008 14671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9ffec90c144f738640e6b0d7319b4c" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42277 } }
I20260808 17:02:36.340109 14671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:36.340142 14671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:36.340188 14671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:36.340773 14671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9ffec90c144f738640e6b0d7319b4c" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42277 } }
I20260808 17:02:36.340787 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:36.335589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59974 (local address 127.30.194.62:42277)
0808 17:02:36.335788 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:36.335796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:36.335827 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:02:36.336346 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:36.336355 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:36.336432 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:36.336573 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:36.336743 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.337946 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.338799 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.339422 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.339550 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:36.340111 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:36.340116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:36.340116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:36.340328 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:36.340363 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:36.340375 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:36.340430 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:36.340531 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 17:02:36.340593 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260808 17:02:36.340873 14671 leader_election.cc:302] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d9ffec90c144f738640e6b0d7319b4c; no voters:
I20260808 17:02:36.341012 14671 leader_election.cc:288] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:36.341070 14678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:36.341277 14671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:36.341657 14678 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [term 1 LEADER]: Becoming Leader. State: Replica: 4d9ffec90c144f738640e6b0d7319b4c, State: Running, Role: LEADER
I20260808 17:02:36.341776 14678 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9ffec90c144f738640e6b0d7319b4c" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42277 } }
I20260808 17:02:36.343113 14681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d9ffec90c144f738640e6b0d7319b4c. Latest consensus state: current_term: 1 leader_uuid: "4d9ffec90c144f738640e6b0d7319b4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9ffec90c144f738640e6b0d7319b4c" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42277 } } }
I20260808 17:02:36.343190 14681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [sys.catalog]: This master's current role is: LEADER
W20260808 17:02:36.343315 14699 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 17:02:36.343366 14680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d9ffec90c144f738640e6b0d7319b4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9ffec90c144f738640e6b0d7319b4c" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42277 } } }
W20260808 17:02:36.343426 14699 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:02:36.343439 14680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:36.343516 14702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:36.343690 14702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:36.345134 14702 catalog_manager.cc:1347] Generated new cluster ID: d35f19bf32814bbeb72ed7e472daf930
I20260808 17:02:36.345191 14702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:36.353288 14702 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:36.353792 14702 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:36.358553 14702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d9ffec90c144f738640e6b0d7319b4c: Generated new TSK 0
I20260808 17:02:36.358723 14702 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:02:36.360932 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42277
--builtin_ntp_servers=127.30.194.20:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 17:02:36.440428 14705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:36.440618 14705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:36.440646 14705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:36.441839 14705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:36.441877 14705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:36.441910 14705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:36.443279 14705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:36.443516 14705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:36.443785 14705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:36.448727 14712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:36.448841 14711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:36.448762 14714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:36.449349 14705 server_base.cc:1034] running on GCE node
I20260808 17:02:36.449584 14705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:36.449865 14705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:36.451056 14705 hybrid_clock.cc:648] HybridClock initialized: now 1786208556451032 us; error 34 us; skew 500 ppm
I20260808 17:02:36.452543 14705 webserver.cc:460] Webserver started at http://127.30.194.1:40499/ using document root <none> and password file <none>
I20260808 17:02:36.452847 14705 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:36.452924 14705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:36.453070 14705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:36.454478 14705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "0281c37490f44d2abb0b98fa0972ff68"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "4b379317eef6b6c933ad31097de759cb"
server_key_iv: "898a546b73f45e3990111449c30d6659"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.454924 14705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "0281c37490f44d2abb0b98fa0972ff68"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "4b379317eef6b6c933ad31097de759cb"
server_key_iv: "898a546b73f45e3990111449c30d6659"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.456670 14705 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 17:02:36.457607 14721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.457779 14705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.457885 14705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "0281c37490f44d2abb0b98fa0972ff68"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "4b379317eef6b6c933ad31097de759cb"
server_key_iv: "898a546b73f45e3990111449c30d6659"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.457967 14705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:36.474769 14705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:36.475078 14705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:36.475219 14705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:36.475443 14705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:36.475790 14705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:36.475833 14705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.475873 14705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:36.475901 14705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.483078 14705 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:41705
I20260808 17:02:36.483114 14898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:41705 every 8 connection(s)
I20260808 17:02:36.483459 14705 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:02:36.486263 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 14705
I20260808 17:02:36.486378 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:02:36.486595 31496 external_mini_cluster.cc:1428] Setting key 611db93dc4dc9ce319871b2357cd73e1
I20260808 17:02:36.487532 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42277
--builtin_ntp_servers=127.30.194.20:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 17:02:36.488679 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:36.484192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:48991 (local address 127.30.194.62:42277)
0808 17:02:36.484255 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:36.484257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:36.484472 (+ 215us) server_negotiation.cc:407] Connection header received
0808 17:02:36.484959 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:36.484960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:36.484972 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:36.485029 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:36.485692 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.486203 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.487322 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.487510 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.487560 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:36.488448 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:36.488449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:36.488450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:36.488454 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:36.488469 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:36.488473 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:36.488524 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:36.488597 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 17:02:36.488622 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:02:36.488922 14903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:36.484179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:42277 (local address 127.30.194.1:48991)
0808 17:02:36.484365 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:36.484400 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:36.484866 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:36.485107 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:36.485112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:36.485123 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:36.485604 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:36.485609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.486295 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.486296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:36.487240 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:36.487243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.487592 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.487594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:36.487746 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:36.488280 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:36.488281 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:36.488318 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:36.488344 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:36.488534 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:36.488539 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:36.488542 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:36.488600 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:02:36.488704 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260808 17:02:36.489444 14900 heartbeater.cc:347] Connected to a master server at 127.30.194.62:42277
I20260808 17:02:36.489538 14900 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:36.489712 14900 heartbeater.cc:511] Master 127.30.194.62:42277 requested a full tablet report, sending...
I20260808 17:02:36.490212 14612 ts_manager.cc:194] Registered new tserver with Master: 0281c37490f44d2abb0b98fa0972ff68 (127.30.194.1:41705)
I20260808 17:02:36.490804 14612 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.1:48991
W20260808 17:02:36.568041 14904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:36.568219 14904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:36.568238 14904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:36.569367 14904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:36.569406 14904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:36.569429 14904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:36.570706 14904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:36.570912 14904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:36.571161 14904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:36.575816 14911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:36.575805 14914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:36.575832 14904 server_base.cc:1034] running on GCE node
W20260808 17:02:36.575831 14912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:36.576198 14904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:36.576419 14904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:36.577553 14904 hybrid_clock.cc:648] HybridClock initialized: now 1786208556577525 us; error 39 us; skew 500 ppm
I20260808 17:02:36.578656 14904 webserver.cc:460] Webserver started at http://127.30.194.2:41349/ using document root <none> and password file <none>
I20260808 17:02:36.578855 14904 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:36.578912 14904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:36.579041 14904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:36.580015 14904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "8d6d2b8177f943a7b257b5a248e56e79"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "297952627efa4f5b9d62bbf202690c3e"
server_key_iv: "1d91f930b2779236109f9c351ad5db36"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.580328 14904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "8d6d2b8177f943a7b257b5a248e56e79"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "297952627efa4f5b9d62bbf202690c3e"
server_key_iv: "1d91f930b2779236109f9c351ad5db36"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.581559 14904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:02:36.582231 14925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.582382 14904 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:36.582453 14904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "8d6d2b8177f943a7b257b5a248e56e79"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "297952627efa4f5b9d62bbf202690c3e"
server_key_iv: "1d91f930b2779236109f9c351ad5db36"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.582518 14904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:36.590389 14904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:36.590585 14904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:36.590672 14904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:36.590821 14904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:36.591054 14904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:36.591082 14904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.591106 14904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:36.591122 14904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.596385 14904 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:44249
I20260808 17:02:36.596418 15109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:44249 every 8 connection(s)
I20260808 17:02:36.596675 14904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:02:36.601261 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:36.597336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:35595 (local address 127.30.194.62:42277)
0808 17:02:36.597398 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:36.597400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:36.597558 (+ 158us) server_negotiation.cc:407] Connection header received
0808 17:02:36.597977 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:36.597978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:36.597991 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:36.598048 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:36.598644 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.599133 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.600027 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.600273 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.600331 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:36.600960 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:36.600961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:36.600962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:36.600966 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:36.600981 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:36.600986 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:36.601030 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:36.601154 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 17:02:36.601183 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:02:36.601449 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:36.597277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:42277 (local address 127.30.194.2:35595)
0808 17:02:36.597457 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:36.597485 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:36.597893 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:36.598110 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:36.598113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:36.598121 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:36.598563 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:36.598567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.599224 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.599225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:36.599942 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:36.599944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.600345 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.600346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:36.600467 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:36.600841 (+ 374us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:36.600841 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:36.600865 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:36.600885 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:36.601036 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:36.601041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:36.601044 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:36.601146 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 17:02:36.601273 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260808 17:02:36.601838 15113 heartbeater.cc:347] Connected to a master server at 127.30.194.62:42277
I20260808 17:02:36.601908 15113 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:36.602058 15113 heartbeater.cc:511] Master 127.30.194.62:42277 requested a full tablet report, sending...
I20260808 17:02:36.602380 14612 ts_manager.cc:194] Registered new tserver with Master: 8d6d2b8177f943a7b257b5a248e56e79 (127.30.194.2:44249)
I20260808 17:02:36.602705 14612 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.2:35595
I20260808 17:02:36.602918 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 14904
I20260808 17:02:36.603013 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:02:36.603181 31496 external_mini_cluster.cc:1428] Setting key 0353784854d06571b74891d828432614
I20260808 17:02:36.603989 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42277
--builtin_ntp_servers=127.30.194.20:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 17:02:36.685246 15117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:36.685426 15117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:36.685447 15117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:36.686960 15117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:36.687009 15117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:36.687036 15117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:36.688345 15117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:36.688557 15117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:36.688827 15117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:36.693467 15124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:36.693464 15123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:36.693446 15127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:36.693665 15117 server_base.cc:1034] running on GCE node
I20260808 17:02:36.693902 15117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:36.694135 15117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:36.695266 15117 hybrid_clock.cc:648] HybridClock initialized: now 1786208556695253 us; error 28 us; skew 500 ppm
I20260808 17:02:36.696426 15117 webserver.cc:460] Webserver started at http://127.30.194.3:38187/ using document root <none> and password file <none>
I20260808 17:02:36.696645 15117 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:36.696691 15117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:36.696808 15117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:36.697810 15117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "0063c38e09b342278ff525d1823f8614"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "8a3a69e1d6769759ef84a505dbba5898"
server_key_iv: "e41d9e9845067bec8d4168b062c0e7d8"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.698130 15117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "0063c38e09b342278ff525d1823f8614"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "8a3a69e1d6769759ef84a505dbba5898"
server_key_iv: "e41d9e9845067bec8d4168b062c0e7d8"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.699286 15117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:02:36.699954 15137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.700116 15117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:36.700196 15117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "0063c38e09b342278ff525d1823f8614"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "8a3a69e1d6769759ef84a505dbba5898"
server_key_iv: "e41d9e9845067bec8d4168b062c0e7d8"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.700253 15117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:36.714437 15117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:36.714663 15117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:36.714772 15117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:36.714968 15117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:36.715250 15117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:36.715283 15117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.715309 15117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:36.715333 15117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.720796 15117 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:38851
I20260808 17:02:36.720875 15309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:38851 every 8 connection(s)
I20260808 17:02:36.721185 15117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:02:36.725661 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:36.721826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:53337 (local address 127.30.194.62:42277)
0808 17:02:36.721875 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:36.721877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:36.722089 (+ 212us) server_negotiation.cc:407] Connection header received
0808 17:02:36.722531 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:36.722532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:36.722543 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:36.722600 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:36.723169 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.723648 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.724520 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.724704 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.724752 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:36.725450 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:36.725451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:36.725451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:36.725456 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:36.725472 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:36.725477 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:36.725528 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:36.725575 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 17:02:36.725601 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:02:36.725813 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:36.721790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:42277 (local address 127.30.194.3:53337)
0808 17:02:36.721973 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:36.722010 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:36.722444 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:36.722650 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:36.722652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:36.722660 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:36.723109 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:36.723112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.723718 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.723718 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:36.724460 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:36.724462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:36.724758 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:36.724759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:36.724855 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:36.725313 (+ 458us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:36.725314 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:36.725339 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:36.725359 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:36.725530 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:36.725532 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:36.725534 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:02:36.725576 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 17:02:36.725647 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260808 17:02:36.726294 15312 heartbeater.cc:347] Connected to a master server at 127.30.194.62:42277
I20260808 17:02:36.726387 15312 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:36.726559 15312 heartbeater.cc:511] Master 127.30.194.62:42277 requested a full tablet report, sending...
I20260808 17:02:36.726938 14612 ts_manager.cc:194] Registered new tserver with Master: 0063c38e09b342278ff525d1823f8614 (127.30.194.3:38851)
I20260808 17:02:36.727272 14612 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.3:53337
I20260808 17:02:36.728924 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 15117
I20260808 17:02:36.729008 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:02:36.729207 31496 external_mini_cluster.cc:1428] Setting key a01043cbfc5cbd73c5ae8f2ff19072b2
I20260808 17:02:36.730026 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 17:02:36.731631 14675 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:36.731218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43464 (local address 127.30.194.62:42277)
0808 17:02:36.731257 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:36.731259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:36.731364 (+ 105us) server_negotiation.cc:407] Connection header received
0808 17:02:36.731423 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:36.731424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:36.731432 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:36.731480 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:36.731584 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43464: BlockingRecv error: recv got EOF from 127.0.0.1:43464 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:02:36.731729 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 14705
I20260808 17:02:36.738502 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 14904
I20260808 17:02:36.750304 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 15117
I20260808 17:02:36.757900 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 14565
2026-08-08T17:02:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (571 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T17:02:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:36Z Disabled control of system clock
I20260808 17:02:36.792378 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:33549
--webserver_interface=127.30.194.62
--webserver_port=0
--builtin_ntp_servers=127.30.194.20:35923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:33549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 17:02:36.873437 15328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:36.873613 15328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:36.873632 15328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:36.874689 15328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:36.874722 15328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:36.874734 15328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:36.874744 15328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:36.874755 15328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:36.876010 15328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:35923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:33549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:33549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:36.876185 15328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:36.876412 15328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:36.880718 15341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:36.880731 15338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:36.880829 15328 server_base.cc:1034] running on GCE node
W20260808 17:02:36.880731 15339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:36.881203 15328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:36.881485 15328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:36.882620 15328 hybrid_clock.cc:648] HybridClock initialized: now 1786208556882602 us; error 38 us; skew 500 ppm
I20260808 17:02:36.883738 15328 webserver.cc:460] Webserver started at http://127.30.194.62:37889/ using document root <none> and password file <none>
I20260808 17:02:36.883951 15328 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:36.883994 15328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:36.884114 15328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:36.885124 15328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "c10d3651832e4c80bcda6584550cee87"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "abdf7adfdab83c495a92cd889b8649b6"
server_key_iv: "2123420d0152422f00376c652513bd01"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.885449 15328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "c10d3651832e4c80bcda6584550cee87"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "abdf7adfdab83c495a92cd889b8649b6"
server_key_iv: "2123420d0152422f00376c652513bd01"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.886637 15328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:02:36.887245 15350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:36.887377 15328 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:36.887454 15328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "c10d3651832e4c80bcda6584550cee87"
format_stamp: "Formatted at 2026-08-08 17:02:36 on dist-test-slave-2j7r"
server_key: "abdf7adfdab83c495a92cd889b8649b6"
server_key_iv: "2123420d0152422f00376c652513bd01"
server_key_version: "encryptionkey@0"
I20260808 17:02:36.887516 15328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:36.890641 15328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:36.890863 15328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:36.890974 15328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:36.894623 15328 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:33549
I20260808 17:02:36.894652 15433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:33549 every 8 connection(s)
I20260808 17:02:36.894934 15328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:36.895507 15434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:36.896193 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 15328
I20260808 17:02:36.896284 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:36.896449 31496 external_mini_cluster.cc:1428] Setting key 81f550f5f092166370b8e7a2b1ac639c
I20260808 17:02:36.898423 15434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87: Bootstrap starting.
I20260808 17:02:36.898643 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:36.896908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59460 (local address 127.30.194.62:33549)
0808 17:02:36.897073 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:36.897079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:36.897128 (+ 49us) server_negotiation.cc:407] Connection header received
0808 17:02:36.897511 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:36.897517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:36.897553 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:36.897659 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:36.898255 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:36.898258 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:36.898258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:36.898262 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:36.898301 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:36.898310 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:36.898355 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:36.898431 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 17:02:36.898474 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":51,"threads_started":1}
I20260808 17:02:36.899067 15434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:36.899353 15434 log.cc:824] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:36.899936 15434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87: No bootstrap required, opened a new log
I20260808 17:02:36.901376 15434 raft_consensus.cc:348] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10d3651832e4c80bcda6584550cee87" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33549 } }
I20260808 17:02:36.901587 15434 raft_consensus.cc:374] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:36.901638 15434 raft_consensus.cc:729] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10d3651832e4c80bcda6584550cee87, State: Initialized, Role: FOLLOWER
I20260808 17:02:36.901753 15434 consensus_queue.cc:260] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10d3651832e4c80bcda6584550cee87" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33549 } }
I20260808 17:02:36.901837 15434 raft_consensus.cc:388] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:36.901876 15434 raft_consensus.cc:482] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:36.901921 15434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:36.902384 15434 raft_consensus.cc:504] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10d3651832e4c80bcda6584550cee87" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33549 } }
I20260808 17:02:36.902486 15434 leader_election.cc:302] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c10d3651832e4c80bcda6584550cee87; no voters:
I20260808 17:02:36.902642 15434 leader_election.cc:288] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:36.902868 15434 sys_catalog.cc:564] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:36.902876 15443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:36.903002 15443 raft_consensus.cc:686] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [term 1 LEADER]: Becoming Leader. State: Replica: c10d3651832e4c80bcda6584550cee87, State: Running, Role: LEADER
I20260808 17:02:36.903095 15443 consensus_queue.cc:237] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10d3651832e4c80bcda6584550cee87" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33549 } }
I20260808 17:02:36.903405 15448 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c10d3651832e4c80bcda6584550cee87. Latest consensus state: current_term: 1 leader_uuid: "c10d3651832e4c80bcda6584550cee87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10d3651832e4c80bcda6584550cee87" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33549 } } }
I20260808 17:02:36.903486 15448 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:36.903666 15458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:36.903702 15445 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c10d3651832e4c80bcda6584550cee87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10d3651832e4c80bcda6584550cee87" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33549 } } }
I20260808 17:02:36.904062 15445 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:36.904158 15458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:36.905619 15458 catalog_manager.cc:1347] Generated new cluster ID: e0dc03dea6b94e9382c4f9f8f56dda1e
I20260808 17:02:36.905673 15458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:02:36.923892 15458 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:36.924047 15458 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:36.935070 15458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c10d3651832e4c80bcda6584550cee87: Generated new TSK 0
I20260808 17:02:36.935250 15458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:02:36.936915 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33549
--builtin_ntp_servers=127.30.194.20:35923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 17:02:37.017951 15480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:37.018133 15480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:37.018153 15480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:37.019271 15480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:37.019310 15480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:37.019333 15480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:37.020615 15480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:35923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:37.020833 15480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:37.021113 15480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:37.025709 15488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:37.025719 15491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:37.025906 15480 server_base.cc:1034] running on GCE node
W20260808 17:02:37.025736 15495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:37.026167 15480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:37.026419 15480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:37.027583 15480 hybrid_clock.cc:648] HybridClock initialized: now 1786208557027572 us; error 27 us; skew 500 ppm
I20260808 17:02:37.028707 15480 webserver.cc:460] Webserver started at http://127.30.194.1:45113/ using document root <none> and password file <none>
I20260808 17:02:37.028880 15480 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:37.028918 15480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:37.029008 15480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:37.030038 15480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "ca621f5a0e1e441fae475aceb80b8ce8"
format_stamp: "Formatted at 2026-08-08 17:02:37 on dist-test-slave-2j7r"
server_key: "85d4796d5de4c3e7204421e9111b3129"
server_key_iv: "334018d70bf72a9f38d2a42c469935b8"
server_key_version: "encryptionkey@0"
I20260808 17:02:37.030331 15480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "ca621f5a0e1e441fae475aceb80b8ce8"
format_stamp: "Formatted at 2026-08-08 17:02:37 on dist-test-slave-2j7r"
server_key: "85d4796d5de4c3e7204421e9111b3129"
server_key_iv: "334018d70bf72a9f38d2a42c469935b8"
server_key_version: "encryptionkey@0"
I20260808 17:02:37.031484 15480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:37.032156 15506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:37.032325 15480 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:37.032408 15480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "ca621f5a0e1e441fae475aceb80b8ce8"
format_stamp: "Formatted at 2026-08-08 17:02:37 on dist-test-slave-2j7r"
server_key: "85d4796d5de4c3e7204421e9111b3129"
server_key_iv: "334018d70bf72a9f38d2a42c469935b8"
server_key_version: "encryptionkey@0"
I20260808 17:02:37.032472 15480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:37.035732 15480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:37.035965 15480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:37.036077 15480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:37.036270 15480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:37.036535 15480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:37.036567 15480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:37.036592 15480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:37.036614 15480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:37.041917 15480 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:36917
I20260808 17:02:37.041954 15683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:36917 every 8 connection(s)
I20260808 17:02:37.042258 15480 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:02:37.044729 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:37.042899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:39187 (local address 127.30.194.62:33549)
0808 17:02:37.042945 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:37.042947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:37.043134 (+ 187us) server_negotiation.cc:407] Connection header received
0808 17:02:37.043546 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:37.043547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:37.043551 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:37.043586 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:37.044576 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:37.044576 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:37.044576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:37.044579 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:37.044587 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:37.044590 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:37.044622 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:37.044670 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 17:02:37.044679 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:02:37.044889 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:37.042863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:33549 (local address 127.30.194.1:39187)
0808 17:02:37.043027 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:37.043063 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:37.043461 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:37.043639 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:37.043641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:37.043649 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:37.044466 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:37.044467 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:37.044493 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:37.044518 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:37.044622 (+ 104us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:37.044624 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:37.044626 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:02:37.044673 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:02:37.044715 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260808 17:02:37.045348 15684 heartbeater.cc:347] Connected to a master server at 127.30.194.62:33549
I20260808 17:02:37.045428 15684 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:37.045560 15684 heartbeater.cc:511] Master 127.30.194.62:33549 requested a full tablet report, sending...
I20260808 17:02:37.046010 15377 ts_manager.cc:194] Registered new tserver with Master: ca621f5a0e1e441fae475aceb80b8ce8 (127.30.194.1:36917)
I20260808 17:02:37.052366 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 15480
I20260808 17:02:37.052455 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:02:37.052644 31496 external_mini_cluster.cc:1428] Setting key affe534777cee9cd0a6e0bc33b311b03
I20260808 17:02:37.053483 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33549
--builtin_ntp_servers=127.30.194.20:35923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 17:02:37.132771 15689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:37.132949 15689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:37.132969 15689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:37.134116 15689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:37.134155 15689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:37.134179 15689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:37.135556 15689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:35923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:37.135773 15689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:37.136051 15689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:37.140550 15697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:37.140573 15701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:37.140589 15689 server_base.cc:1034] running on GCE node
W20260808 17:02:37.140581 15696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:37.140923 15689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:37.141173 15689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:37.142342 15689 hybrid_clock.cc:648] HybridClock initialized: now 1786208557142324 us; error 31 us; skew 500 ppm
I20260808 17:02:37.143553 15689 webserver.cc:460] Webserver started at http://127.30.194.2:33521/ using document root <none> and password file <none>
I20260808 17:02:37.143761 15689 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:37.143805 15689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:37.143929 15689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:37.144887 15689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "1de0ef6b26064ef79d3cde37bc8b32c4"
format_stamp: "Formatted at 2026-08-08 17:02:37 on dist-test-slave-2j7r"
server_key: "06bdca6f804785ad9fb90408d165d49f"
server_key_iv: "8417bd50b2def64426bab6d768400400"
server_key_version: "encryptionkey@0"
I20260808 17:02:37.145221 15689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "1de0ef6b26064ef79d3cde37bc8b32c4"
format_stamp: "Formatted at 2026-08-08 17:02:37 on dist-test-slave-2j7r"
server_key: "06bdca6f804785ad9fb90408d165d49f"
server_key_iv: "8417bd50b2def64426bab6d768400400"
server_key_version: "encryptionkey@0"
I20260808 17:02:37.146380 15689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:02:37.146986 15712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:37.147120 15689 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:02:37.147181 15689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "1de0ef6b26064ef79d3cde37bc8b32c4"
format_stamp: "Formatted at 2026-08-08 17:02:37 on dist-test-slave-2j7r"
server_key: "06bdca6f804785ad9fb90408d165d49f"
server_key_iv: "8417bd50b2def64426bab6d768400400"
server_key_version: "encryptionkey@0"
I20260808 17:02:37.147244 15689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:37.150354 15689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:37.150559 15689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:37.150665 15689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:37.150835 15689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:37.151115 15689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:37.151146 15689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:37.151175 15689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:37.151196 15689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:37.156217 15689 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:37311
I20260808 17:02:37.156287 15899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:37311 every 8 connection(s)
I20260808 17:02:37.156525 15689 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:02:37.158175 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 15689
I20260808 17:02:37.158283 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:02:37.158484 31496 external_mini_cluster.cc:1428] Setting key 2c97e045aa6daf87b5932e22fb4ffeb5
I20260808 17:02:37.159361 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:37.157147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:50931 (local address 127.30.194.62:33549)
0808 17:02:37.157184 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:37.157186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:37.157430 (+ 244us) server_negotiation.cc:407] Connection header received
0808 17:02:37.157849 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:37.157850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:37.157857 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:37.157940 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:37.159134 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:37.159136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:37.159136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:37.159140 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:37.159156 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:37.159161 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:37.159222 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:37.159278 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 17:02:37.159291 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:02:37.159454 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33549
--builtin_ntp_servers=127.30.194.20:35923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 17:02:37.159511 15903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:37.157140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:33549 (local address 127.30.194.2:50931)
0808 17:02:37.157322 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:37.157351 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:37.157747 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:37.158024 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:37.158028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:37.158037 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:37.158986 (+ 949us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:37.158986 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:37.159021 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:37.159051 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:37.159220 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:37.159224 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:37.159227 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:37.159283 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 17:02:37.159336 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260808 17:02:37.160039 15900 heartbeater.cc:347] Connected to a master server at 127.30.194.62:33549
I20260808 17:02:37.160131 15900 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:37.160269 15900 heartbeater.cc:511] Master 127.30.194.62:33549 requested a full tablet report, sending...
I20260808 17:02:37.160570 15377 ts_manager.cc:194] Registered new tserver with Master: 1de0ef6b26064ef79d3cde37bc8b32c4 (127.30.194.2:37311)
W20260808 17:02:37.239140 15905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:37.239331 15905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:37.239351 15905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:37.240463 15905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:37.240500 15905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:37.240527 15905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:37.241881 15905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:35923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:37.242089 15905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:37.242383 15905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:37.247052 15912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:37.247052 15916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:37.247123 15905 server_base.cc:1034] running on GCE node
W20260808 17:02:37.247105 15910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:37.247489 15905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:37.247713 15905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:37.248849 15905 hybrid_clock.cc:648] HybridClock initialized: now 1786208557248837 us; error 30 us; skew 500 ppm
I20260808 17:02:37.250017 15905 webserver.cc:460] Webserver started at http://127.30.194.3:35967/ using document root <none> and password file <none>
I20260808 17:02:37.250239 15905 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:37.250283 15905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:37.250412 15905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:37.251451 15905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "b9e4ad5df11d46e6b57580c547cfbd6b"
format_stamp: "Formatted at 2026-08-08 17:02:37 on dist-test-slave-2j7r"
server_key: "c67770d1e47bd976db4c274b46725d5f"
server_key_iv: "cb33e3ca642a537cb6d4deebedad363e"
server_key_version: "encryptionkey@0"
I20260808 17:02:37.251770 15905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "b9e4ad5df11d46e6b57580c547cfbd6b"
format_stamp: "Formatted at 2026-08-08 17:02:37 on dist-test-slave-2j7r"
server_key: "c67770d1e47bd976db4c274b46725d5f"
server_key_iv: "cb33e3ca642a537cb6d4deebedad363e"
server_key_version: "encryptionkey@0"
I20260808 17:02:37.252947 15905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:02:37.253652 15923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:37.253777 15905 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:37.253845 15905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "b9e4ad5df11d46e6b57580c547cfbd6b"
format_stamp: "Formatted at 2026-08-08 17:02:37 on dist-test-slave-2j7r"
server_key: "c67770d1e47bd976db4c274b46725d5f"
server_key_iv: "cb33e3ca642a537cb6d4deebedad363e"
server_key_version: "encryptionkey@0"
I20260808 17:02:37.253906 15905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:37.257023 15905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:37.257268 15905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:37.257390 15905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:37.257578 15905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:37.257843 15905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:37.257875 15905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:37.257905 15905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:37.257927 15905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:37.263294 15905 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:46839
I20260808 17:02:37.263350 16103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:46839 every 8 connection(s)
I20260808 17:02:37.263618 15905 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:02:37.264364 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 15905
I20260808 17:02:37.264456 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:02:37.264616 31496 external_mini_cluster.cc:1428] Setting key ec5d5afbce51f35cf1660d616c587775
I20260808 17:02:37.266798 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:37.264291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:44385 (local address 127.30.194.62:33549)
0808 17:02:37.264353 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:37.264355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:37.264916 (+ 561us) server_negotiation.cc:407] Connection header received
0808 17:02:37.265450 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:37.265451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:37.265458 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:37.265510 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:37.266586 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:37.266587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:37.266587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:02:37.266591 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:37.266606 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:02:37.266625 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:37.266669 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:37.266736 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 17:02:37.266746 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:02:37.266963 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:37.264236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:33549 (local address 127.30.194.3:44385)
0808 17:02:37.264816 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:37.264851 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:37.265349 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:37.265579 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:37.265583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:37.265593 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:37.266450 (+ 857us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:02:37.266451 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:37.266482 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:02:37.266515 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:37.266677 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:37.266682 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:37.266684 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:02:37.266742 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:02:37.266788 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":69,"threads_started":1}
I20260808 17:02:37.267369 16104 heartbeater.cc:347] Connected to a master server at 127.30.194.62:33549
I20260808 17:02:37.267446 16104 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:37.267577 16104 heartbeater.cc:511] Master 127.30.194.62:33549 requested a full tablet report, sending...
I20260808 17:02:37.267894 15377 ts_manager.cc:194] Registered new tserver with Master: b9e4ad5df11d46e6b57580c547cfbd6b (127.30.194.3:46839)
I20260808 17:02:37.276274 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 17:02:37.277954 15438 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:02:37.277536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59468 (local address 127.30.194.62:33549)
0808 17:02:37.277575 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:37.277578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:37.277661 (+ 83us) server_negotiation.cc:407] Connection header received
0808 17:02:37.277734 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:37.277734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:37.277739 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:37.277772 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:37.277893 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59468: BlockingRecv error: recv got EOF from 127.0.0.1:59468 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:02:37.278045 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 15480
I20260808 17:02:37.305399 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 15689
I20260808 17:02:37.317144 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 15905
I20260808 17:02:37.322227 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 15328
2026-08-08T17:02:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (558 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:02:37 dist-test-slave-2j7r krb5kdc[16120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:02:37 dist-test-slave-2j7r krb5kdc[16120](info): set up 2 sockets
Aug 08 17:02:37 dist-test-slave-2j7r krb5kdc[16120](info): commencing operation
krb5kdc: starting...
W20260808 17:02:38.471230 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:02:38 dist-test-slave-2j7r krb5kdc[16120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:02:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:02:38.580538 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:42287
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:42287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:38.659198 16137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:38.659399 16137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:38.659427 16137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:38.660529 16137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:02:38.660568 16137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:38.660585 16137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:38.660601 16137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:02:38.660615 16137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:02:38.662039 16137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:42287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:42287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:38.662278 16137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:38.662469 16137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:38.667191 16152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:38.667224 16148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:38.667189 16154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:38.667341 16137 server_base.cc:1034] running on GCE node
I20260808 17:02:38.667644 16137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:38.667927 16137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:38.669062 16137 hybrid_clock.cc:648] HybridClock initialized: now 1786208558669043 us; error 33 us; skew 500 ppm
Aug 08 17:02:38 dist-test-slave-2j7r krb5kdc[16120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:38.671439 16137 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:02:38.671816 16137 webserver.cc:460] Webserver started at http://127.30.194.62:40391/ using document root <none> and password file <none>
I20260808 17:02:38.672014 16137 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:38.672060 16137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:38.672178 16137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:38.673183 16137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "27cee1970aff4b44aa94d71105aa51a2"
format_stamp: "Formatted at 2026-08-08 17:02:38 on dist-test-slave-2j7r"
server_key: "350b5d49ed47816d9fc063d69287e259"
server_key_iv: "18bc5fd5fc5c0f998c6d0e7afc1eab2d"
server_key_version: "encryptionkey@0"
I20260808 17:02:38.673504 16137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "27cee1970aff4b44aa94d71105aa51a2"
format_stamp: "Formatted at 2026-08-08 17:02:38 on dist-test-slave-2j7r"
server_key: "350b5d49ed47816d9fc063d69287e259"
server_key_iv: "18bc5fd5fc5c0f998c6d0e7afc1eab2d"
server_key_version: "encryptionkey@0"
I20260808 17:02:38.674741 16137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:38.675426 16165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:38.675571 16137 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:38.675655 16137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "27cee1970aff4b44aa94d71105aa51a2"
format_stamp: "Formatted at 2026-08-08 17:02:38 on dist-test-slave-2j7r"
server_key: "350b5d49ed47816d9fc063d69287e259"
server_key_iv: "18bc5fd5fc5c0f998c6d0e7afc1eab2d"
server_key_version: "encryptionkey@0"
I20260808 17:02:38.675714 16137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:38.698838 16137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:38.700629 16137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:38.700757 16137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:38.704581 16137 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:42287
I20260808 17:02:38.704632 16243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:42287 every 8 connection(s)
I20260808 17:02:38.704955 16137 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:02:38.705977 16244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:38.708343 16244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2: Bootstrap starting.
I20260808 17:02:38.708928 16244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:38.709244 16244 log.cc:824] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:38.709934 16244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2: No bootstrap required, opened a new log
I20260808 17:02:38.711551 16244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cee1970aff4b44aa94d71105aa51a2" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42287 } }
I20260808 17:02:38.711700 16244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:38.711745 16244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27cee1970aff4b44aa94d71105aa51a2, State: Initialized, Role: FOLLOWER
I20260808 17:02:38.711889 16244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cee1970aff4b44aa94d71105aa51a2" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42287 } }
I20260808 17:02:38.711958 16244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:02:38.711984 16244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:02:38.712035 16244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:38.712811 16244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cee1970aff4b44aa94d71105aa51a2" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42287 } }
I20260808 17:02:38.712910 16244 leader_election.cc:302] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27cee1970aff4b44aa94d71105aa51a2; no voters:
I20260808 17:02:38.713076 16244 leader_election.cc:288] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:02:38.713129 16248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:38.713272 16248 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [term 1 LEADER]: Becoming Leader. State: Replica: 27cee1970aff4b44aa94d71105aa51a2, State: Running, Role: LEADER
I20260808 17:02:38.713356 16244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:02:38.713419 16248 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cee1970aff4b44aa94d71105aa51a2" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42287 } }
I20260808 17:02:38.713769 16249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27cee1970aff4b44aa94d71105aa51a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cee1970aff4b44aa94d71105aa51a2" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42287 } } }
I20260808 17:02:38.713778 16251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27cee1970aff4b44aa94d71105aa51a2. Latest consensus state: current_term: 1 leader_uuid: "27cee1970aff4b44aa94d71105aa51a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cee1970aff4b44aa94d71105aa51a2" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42287 } } }
I20260808 17:02:38.713977 16251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:38.714219 16249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2 [sys.catalog]: This master's current role is: LEADER
I20260808 17:02:38.714309 16262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:02:38.714730 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 16137
I20260808 17:02:38.714812 16262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:02:38.714830 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:02:38.715018 31496 external_mini_cluster.cc:1428] Setting key 1f217763c76dab47b5ea49fcb8adc873
I20260808 17:02:38.716881 16262 catalog_manager.cc:1347] Generated new cluster ID: 7b064794f0ee49c4a536b91da05cd98b
I20260808 17:02:38.716934 16262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:02:38 dist-test-slave-2j7r krb5kdc[16120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:38.726076 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:38.715425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.30.194.62:42287)
0808 17:02:38.715583 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:38.715595 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:38.715636 (+ 41us) server_negotiation.cc:407] Connection header received
0808 17:02:38.715773 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:38.715780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:38.715850 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:38.715980 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:38.716561 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:38.717951 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:38.718993 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:38.719589 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:38.719682 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:38.722474 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:38.722479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:38.722480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:38.722493 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:38.724440 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:38.724805 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:38.724808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:38.724809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:38.724933 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:38.725147 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:38.725148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:38.725149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:38.725462 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:38.725629 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:38.725709 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 17:02:38.725847 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260808 17:02:38.731134 16262 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:02:38.731813 16262 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:02:38.744361 16262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2: Generated new TSK 0
I20260808 17:02:38.744521 16262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:02:38.768601 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42287
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:38.848387 16295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:38.848582 16295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:38.848604 16295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:38.849722 16295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:38.849764 16295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:38.849792 16295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:02:38.851176 16295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:38.851373 16295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:38.851545 16295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:38.856279 16309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:38.856375 16311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:38.856419 16295 server_base.cc:1034] running on GCE node
W20260808 17:02:38.856488 16308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:38.856673 16295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:38.856886 16295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:38.858018 16295 hybrid_clock.cc:648] HybridClock initialized: now 1786208558857992 us; error 44 us; skew 500 ppm
Aug 08 17:02:38 dist-test-slave-2j7r krb5kdc[16120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:38.860349 16295 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:02:38.860757 16295 webserver.cc:460] Webserver started at http://127.30.194.1:36955/ using document root <none> and password file <none>
I20260808 17:02:38.860936 16295 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:38.860975 16295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:38.861061 16295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:38.862398 16295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "8e99d570bc764bb6b921a99d7ef151d1"
format_stamp: "Formatted at 2026-08-08 17:02:38 on dist-test-slave-2j7r"
server_key: "96353324bd78cbd7f18d1acb409e8bb9"
server_key_iv: "8675e1c7e193d257b29af6ebb4b15a0e"
server_key_version: "encryptionkey@0"
I20260808 17:02:38.862810 16295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "8e99d570bc764bb6b921a99d7ef151d1"
format_stamp: "Formatted at 2026-08-08 17:02:38 on dist-test-slave-2j7r"
server_key: "96353324bd78cbd7f18d1acb409e8bb9"
server_key_iv: "8675e1c7e193d257b29af6ebb4b15a0e"
server_key_version: "encryptionkey@0"
I20260808 17:02:38.864451 16295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:02:38.865288 16322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:38.865737 16295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:02:38.865849 16295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "8e99d570bc764bb6b921a99d7ef151d1"
format_stamp: "Formatted at 2026-08-08 17:02:38 on dist-test-slave-2j7r"
server_key: "96353324bd78cbd7f18d1acb409e8bb9"
server_key_iv: "8675e1c7e193d257b29af6ebb4b15a0e"
server_key_version: "encryptionkey@0"
I20260808 17:02:38.865919 16295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:38.876542 16295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:38.878525 16295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:38.878623 16295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:38.878790 16295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:38.879043 16295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:38.879074 16295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:38.879092 16295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:38.879108 16295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:38.884317 16295 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:40861
I20260808 17:02:38.884371 16497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:40861 every 8 connection(s)
I20260808 17:02:38.884634 16295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:02:38 dist-test-slave-2j7r krb5kdc[16120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:38.892231 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:38.885256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:58161 (local address 127.30.194.62:42287)
0808 17:02:38.885310 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:38.885312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:38.885498 (+ 186us) server_negotiation.cc:407] Connection header received
0808 17:02:38.885919 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:38.885920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:38.885940 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:38.885990 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:38.886792 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:38.887274 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:38.888140 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:38.888319 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:38.888362 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:38.890185 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:38.890186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:38.890186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:38.890193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:38.891233 (+ 1040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:38.891558 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:38.891558 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:38.891559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:38.891605 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:38.891783 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:38.891783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:38.891784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:38.891883 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:38.891926 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:38.892112 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 17:02:38.892169 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 17:02:38.892390 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:38.885237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:42287 (local address 127.30.194.1:58161)
0808 17:02:38.885386 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:38.885424 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:38.885847 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:38.886038 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:38.886041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:38.886560 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:38.886729 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:38.886733 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:38.887335 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:38.887335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:38.888083 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:38.888084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:38.888383 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:38.888384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:38.888479 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:38.888814 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:38.888822 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:38.890103 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:38.891296 (+ 1193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:38.891303 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:38.891304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:38.891502 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:38.891649 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:38.891650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:38.891650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:38.891743 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:38.891923 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:38.891926 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:38.892023 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 17:02:38.892095 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:02:38.892200 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260808 17:02:38.892836 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 16295
I20260808 17:02:38.892833 16499 heartbeater.cc:347] Connected to a master server at 127.30.194.62:42287
I20260808 17:02:38.892928 16499 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:38.892930 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:02:38.893126 16499 heartbeater.cc:511] Master 127.30.194.62:42287 requested a full tablet report, sending...
I20260808 17:02:38.893162 31496 external_mini_cluster.cc:1428] Setting key bc1f190e9752e1fddba730e16ab4a193
I20260808 17:02:38.893620 16187 ts_manager.cc:194] Registered new tserver with Master: 8e99d570bc764bb6b921a99d7ef151d1 (127.30.194.1:40861)
I20260808 17:02:38.894227 16187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:58161
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:02:38.916052 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42287
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:38.993965 16506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:38.994139 16506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:38.994169 16506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:38.995263 16506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:38.995301 16506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:38.995335 16506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:02:38.996695 16506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:38.996915 16506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:38.997133 16506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:39.001607 16516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:39.001703 16518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:39.001716 16520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:39.001858 16506 server_base.cc:1034] running on GCE node
I20260808 17:02:39.002051 16506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:39.002318 16506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:39.003475 16506 hybrid_clock.cc:648] HybridClock initialized: now 1786208559003456 us; error 30 us; skew 500 ppm
Aug 08 17:02:39 dist-test-slave-2j7r krb5kdc[16120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208559, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:39.005899 16506 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:02:39.006309 16506 webserver.cc:460] Webserver started at http://127.30.194.2:40733/ using document root <none> and password file <none>
I20260808 17:02:39.006497 16506 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:39.006546 16506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:39.006675 16506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:39.007649 16506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "b6d3e67f735848ec800f3d75e8847740"
format_stamp: "Formatted at 2026-08-08 17:02:39 on dist-test-slave-2j7r"
server_key: "724895943fc1385f937313309ca3f0f7"
server_key_iv: "e9e29068d78b7d224ec96ea5e62f2c08"
server_key_version: "encryptionkey@0"
I20260808 17:02:39.007992 16506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "b6d3e67f735848ec800f3d75e8847740"
format_stamp: "Formatted at 2026-08-08 17:02:39 on dist-test-slave-2j7r"
server_key: "724895943fc1385f937313309ca3f0f7"
server_key_iv: "e9e29068d78b7d224ec96ea5e62f2c08"
server_key_version: "encryptionkey@0"
I20260808 17:02:39.009277 16506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:02:39.009958 16529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:39.010102 16506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:39.010188 16506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "b6d3e67f735848ec800f3d75e8847740"
format_stamp: "Formatted at 2026-08-08 17:02:39 on dist-test-slave-2j7r"
server_key: "724895943fc1385f937313309ca3f0f7"
server_key_iv: "e9e29068d78b7d224ec96ea5e62f2c08"
server_key_version: "encryptionkey@0"
I20260808 17:02:39.010249 16506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:39.024109 16506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:39.025789 16506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:39.025923 16506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:39.026140 16506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:39.026432 16506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:39.026469 16506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:39.026499 16506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:39.026525 16506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:39.032398 16506 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:44899
I20260808 17:02:39.032454 16701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:44899 every 8 connection(s)
I20260808 17:02:39.032727 16506 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:02:39 dist-test-slave-2j7r krb5kdc[16120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208559, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:39.040206 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 16506
I20260808 17:02:39.040323 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:02:39.040515 31496 external_mini_cluster.cc:1428] Setting key 5862bfbe15eb1275b959391ab689dadd
I20260808 17:02:39.041191 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.033360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:36279 (local address 127.30.194.62:42287)
0808 17:02:39.033407 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:39.033409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:39.033619 (+ 210us) server_negotiation.cc:407] Connection header received
0808 17:02:39.034058 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:39.034059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:39.034068 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:39.034117 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:39.034967 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.035547 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.036458 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.036638 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.036676 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.038601 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:39.038603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:39.038603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:39.038610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:39.039844 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:39.040239 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:39.040239 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:39.040239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:39.040289 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:39.040536 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:39.040537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:39.040537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:39.040657 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:39.040713 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:39.040996 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 17:02:39.041079 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:02:39.041322 16704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.033332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:42287 (local address 127.30.194.2:36279)
0808 17:02:39.033496 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:39.033535 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:39.033978 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:39.034167 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:39.034170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:39.034725 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:39.034896 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.034902 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.035616 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.035617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.036388 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.036391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.036692 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.036693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.036782 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.037257 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:39.037265 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:39.038504 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:39.039907 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:39.039909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:39.039911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:39.040168 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:39.040355 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:39.040356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:39.040357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:39.040471 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:39.040716 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:39.040723 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:39.040868 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 17:02:39.040964 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 17:02:39.041128 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260808 17:02:39.043265 16702 heartbeater.cc:347] Connected to a master server at 127.30.194.62:42287
I20260808 17:02:39.043355 16702 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:39.043521 16702 heartbeater.cc:511] Master 127.30.194.62:42287 requested a full tablet report, sending...
I20260808 17:02:39.043898 16187 ts_manager.cc:194] Registered new tserver with Master: b6d3e67f735848ec800f3d75e8847740 (127.30.194.2:44899)
I20260808 17:02:39.044265 16187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:36279
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:02:39.065717 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42287
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:02:39.144101 16708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:02:39.144270 16708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:02:39.144291 16708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:02:39.145375 16708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:02:39.145412 16708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:02:39.145437 16708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:02:39.146807 16708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:42287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:02:39.146976 16708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:02:39.147138 16708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:02:39.151602 16717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:39.151744 16708 server_base.cc:1034] running on GCE node
W20260808 17:02:39.151610 16714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:02:39.151820 16713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:02:39.151991 16708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:02:39.152212 16708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:02:39.153333 16708 hybrid_clock.cc:648] HybridClock initialized: now 1786208559153326 us; error 26 us; skew 500 ppm
Aug 08 17:02:39 dist-test-slave-2j7r krb5kdc[16120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208559, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:02:39.155618 16708 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:02:39.156001 16708 webserver.cc:460] Webserver started at http://127.30.194.3:36949/ using document root <none> and password file <none>
I20260808 17:02:39.156205 16708 fs_manager.cc:359] Metadata directory not provided
I20260808 17:02:39.156252 16708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:02:39.156376 16708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:02:39.157352 16708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "df028094f95f442cb34bf7ff4c6bfe86"
format_stamp: "Formatted at 2026-08-08 17:02:39 on dist-test-slave-2j7r"
server_key: "41541fbd54f9028351eca87ebe4ace15"
server_key_iv: "5ebdac4677a5b93739a8a62b8482a638"
server_key_version: "encryptionkey@0"
I20260808 17:02:39.157653 16708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "df028094f95f442cb34bf7ff4c6bfe86"
format_stamp: "Formatted at 2026-08-08 17:02:39 on dist-test-slave-2j7r"
server_key: "41541fbd54f9028351eca87ebe4ace15"
server_key_iv: "5ebdac4677a5b93739a8a62b8482a638"
server_key_version: "encryptionkey@0"
I20260808 17:02:39.158839 16708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:02:39.159495 16725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:39.159680 16708 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:02:39.159762 16708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "df028094f95f442cb34bf7ff4c6bfe86"
format_stamp: "Formatted at 2026-08-08 17:02:39 on dist-test-slave-2j7r"
server_key: "41541fbd54f9028351eca87ebe4ace15"
server_key_iv: "5ebdac4677a5b93739a8a62b8482a638"
server_key_version: "encryptionkey@0"
I20260808 17:02:39.159816 16708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:02:39.174026 16708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:02:39.175740 16708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:02:39.175863 16708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:02:39.176052 16708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:02:39.176308 16708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:02:39.176342 16708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:39.176368 16708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:02:39.176386 16708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:02:39.181936 16708 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:39087
I20260808 17:02:39.181969 16892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:39087 every 8 connection(s)
I20260808 17:02:39.182230 16708 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:02:39 dist-test-slave-2j7r krb5kdc[16120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208559, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:02:39.189823 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.182851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:60495 (local address 127.30.194.62:42287)
0808 17:02:39.182908 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:39.182910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:39.183069 (+ 159us) server_negotiation.cc:407] Connection header received
0808 17:02:39.183486 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:39.183487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:39.183496 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:39.183545 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:39.184338 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.184809 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.185731 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.185916 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.185976 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.187709 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:39.187711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:39.187711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:39.187718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:39.188755 (+ 1037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:39.189080 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:39.189100 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:39.189100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:39.189143 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:39.189366 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:39.189367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:39.189367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:39.189469 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:39.189517 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:39.189708 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 17:02:39.189762 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 17:02:39.189994 16896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.182818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:42287 (local address 127.30.194.3:60495)
0808 17:02:39.182958 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:39.182996 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:39.183413 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:39.183594 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:39.183597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:39.184106 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:02:39.184275 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.184279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.184874 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.184874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.185657 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.185660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.185984 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.185985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.186079 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.186428 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:02:39.186435 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:02:39.187630 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:02:39.188815 (+ 1185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:39.188820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:39.188822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:39.189026 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:39.189195 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:02:39.189195 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:02:39.189196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:02:39.189310 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:02:39.189513 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:02:39.189517 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:02:39.189617 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 17:02:39.189688 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:02:39.189796 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260808 17:02:39.190119 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 16708
I20260808 17:02:39.190256 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:02:39.190433 16893 heartbeater.cc:347] Connected to a master server at 127.30.194.62:42287
I20260808 17:02:39.190465 31496 external_mini_cluster.cc:1428] Setting key 6b7e35977ed328a97bc682549460e43f
I20260808 17:02:39.190524 16893 heartbeater.cc:464] Registering TS with master...
I20260808 17:02:39.190682 16893 heartbeater.cc:511] Master 127.30.194.62:42287 requested a full tablet report, sending...
I20260808 17:02:39.191049 16187 ts_manager.cc:194] Registered new tserver with Master: df028094f95f442cb34bf7ff4c6bfe86 (127.30.194.3:39087)
I20260808 17:02:39.191316 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:02:39.191557 16187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:60495
I20260808 17:02:39.197570 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.192530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60206 (local address 127.30.194.62:42287)
0808 17:02:39.192563 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:39.192564 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:39.192758 (+ 194us) server_negotiation.cc:407] Connection header received
0808 17:02:39.192808 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:39.192809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:39.192817 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:39.192858 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:02:39.193448 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.193873 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.194532 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.194691 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.194730 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.195644 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:02:39.195645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:02:39.195645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:02:39.195651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:02:39.196771 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:39.197011 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:39.197011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:39.197012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:39.197056 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:02:39.197210 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:02:39.197210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:02:39.197210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:02:39.197325 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:02:39.197405 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:39.197464 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 17:02:39.197517 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 17:02:39.198863 16187 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:02:39.200307 16187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:02:39.206621 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.202383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:40861 (local address 127.0.0.1:56012)
0808 17:02:39.202850 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:39.202859 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:39.202897 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:39.203165 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:39.203166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:39.203168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:39.203202 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.203204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.204456 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.204457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.205323 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.205325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.206269 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.206270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.206360 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.206381 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 17:02:39.206453 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:02:39.206512 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":98,"thread_start_us":99,"threads_started":1}
I20260808 17:02:39.206621 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.202383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:44899 (local address 127.0.0.1:37332)
0808 17:02:39.202737 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:39.202747 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:39.202802 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:39.203060 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:39.203061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:39.203064 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:39.203183 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.203186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.204105 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.204106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.205431 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.205432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.206267 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.206269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.206360 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.206376 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 17:02:39.206459 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 17:02:39.206514 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":26,"thread_start_us":77,"threads_started":1}
I20260808 17:02:39.207109 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.202510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56012 (local address 127.30.194.1:40861)
0808 17:02:39.203006 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:39.203010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:39.203019 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:39.203056 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:39.203057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:39.203072 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:39.203268 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:39.203356 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.204369 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.205407 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.206157 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.206243 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.206303 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:39.206489 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 17:02:39.207031 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":82,"threads_started":1}
I20260808 17:02:39.207231 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.202667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37332 (local address 127.30.194.2:44899)
0808 17:02:39.202930 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:39.202934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:39.202943 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:02:39.202977 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:39.202979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:39.202993 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:39.203061 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:39.203263 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.204001 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.205499 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.206182 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.206244 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.206313 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:39.206570 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 17:02:39.206614 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":92,"threads_started":1}
I20260808 17:02:39.207278 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.202383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:39087 (local address 127.0.0.1:59312)
0808 17:02:39.202644 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:39.202686 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:39.202754 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:39.202967 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:39.202969 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:39.202971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:39.203043 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.203044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.204348 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.204349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.205414 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.205417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.207087 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.207088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.207134 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.207137 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:02:39.207179 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 17:02:39.207214 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":90,"threads_started":1}
I20260808 17:02:39.207342 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.202578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.30.194.3:39087)
0808 17:02:39.202725 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:39.202730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:39.202742 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:02:39.202854 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:39.202857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:39.202875 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:39.202955 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:39.203129 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.204246 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.206348 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.207012 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.207089 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.207145 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:39.207207 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 17:02:39.207246 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260808 17:02:39.208050 16391 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0762fc43b746b9aec57127ee97d5d0 (DEFAULT_TABLE table=test-table [id=dd22a429b3944576833cac472563d41f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:39.208047 16598 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0762fc43b746b9aec57127ee97d5d0 (DEFAULT_TABLE table=test-table [id=dd22a429b3944576833cac472563d41f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:39.208101 16797 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0762fc43b746b9aec57127ee97d5d0 (DEFAULT_TABLE table=test-table [id=dd22a429b3944576833cac472563d41f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:02:39.208403 16797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0762fc43b746b9aec57127ee97d5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:39.208405 16391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0762fc43b746b9aec57127ee97d5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:39.208405 16598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0762fc43b746b9aec57127ee97d5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:02:39.211105 16915 tablet_bootstrap.cc:492] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: Bootstrap starting.
I20260808 17:02:39.211563 16920 tablet_bootstrap.cc:492] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1: Bootstrap starting.
I20260808 17:02:39.211747 16915 tablet_bootstrap.cc:654] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:39.212265 16920 tablet_bootstrap.cc:654] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:39.212311 16915 log.cc:824] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:39.212884 16920 log.cc:824] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:39.213169 16915 tablet_bootstrap.cc:492] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: No bootstrap required, opened a new log
I20260808 17:02:39.213243 16915 ts_tablet_manager.cc:1397] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:02:39.213251 16918 tablet_bootstrap.cc:492] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: Bootstrap starting.
I20260808 17:02:39.213604 16920 tablet_bootstrap.cc:492] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1: No bootstrap required, opened a new log
I20260808 17:02:39.213663 16920 ts_tablet_manager.cc:1397] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:02:39.213968 16918 tablet_bootstrap.cc:654] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: Neither blocks nor log segments found. Creating new log.
I20260808 17:02:39.214592 16918 log.cc:824] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: Log is configured to *not* fsync() on all Append() calls
I20260808 17:02:39.215219 16915 raft_consensus.cc:348] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 } } peers { permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 } } peers { permanent_uuid: "8e99d570bc764bb6b921a99d7ef151d1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40861 } }
I20260808 17:02:39.215372 16918 tablet_bootstrap.cc:492] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: No bootstrap required, opened a new log
I20260808 17:02:39.215399 16915 raft_consensus.cc:374] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:39.215426 16918 ts_tablet_manager.cc:1397] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:02:39.215433 16915 raft_consensus.cc:729] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df028094f95f442cb34bf7ff4c6bfe86, State: Initialized, Role: FOLLOWER
I20260808 17:02:39.215574 16915 consensus_queue.cc:260] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 } } peers { permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 } } peers { permanent_uuid: "8e99d570bc764bb6b921a99d7ef151d1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40861 } }
I20260808 17:02:39.215641 16920 raft_consensus.cc:348] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 } } peers { permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 } } peers { permanent_uuid: "8e99d570bc764bb6b921a99d7ef151d1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40861 } }
I20260808 17:02:39.215814 16920 raft_consensus.cc:374] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:39.215843 16915 ts_tablet_manager.cc:1428] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:02:39.215855 16920 raft_consensus.cc:729] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e99d570bc764bb6b921a99d7ef151d1, State: Initialized, Role: FOLLOWER
I20260808 17:02:39.215972 16893 heartbeater.cc:503] Master 127.30.194.62:42287 was elected leader, sending a full tablet report...
I20260808 17:02:39.215982 16920 consensus_queue.cc:260] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 } } peers { permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 } } peers { permanent_uuid: "8e99d570bc764bb6b921a99d7ef151d1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40861 } }
I20260808 17:02:39.216256 16499 heartbeater.cc:503] Master 127.30.194.62:42287 was elected leader, sending a full tablet report...
I20260808 17:02:39.216241 16920 ts_tablet_manager.cc:1428] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:02:39.217272 16918 raft_consensus.cc:348] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 } } peers { permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 } } peers { permanent_uuid: "8e99d570bc764bb6b921a99d7ef151d1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40861 } }
I20260808 17:02:39.217422 16918 raft_consensus.cc:374] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:02:39.217458 16918 raft_consensus.cc:729] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d3e67f735848ec800f3d75e8847740, State: Initialized, Role: FOLLOWER
I20260808 17:02:39.217561 16918 consensus_queue.cc:260] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 } } peers { permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 } } peers { permanent_uuid: "8e99d570bc764bb6b921a99d7ef151d1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40861 } }
I20260808 17:02:39.217818 16918 ts_tablet_manager.cc:1428] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:02:39.217814 16702 heartbeater.cc:503] Master 127.30.194.62:42287 was elected leader, sending a full tablet report...
W20260808 17:02:39.283277 16703 tablet.cc:2369] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:02:39.385247 16501 tablet.cc:2369] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:02:39.432730 16894 tablet.cc:2369] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:02:39.595104 16925 raft_consensus.cc:482] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:02:39.595237 16925 raft_consensus.cc:504] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 } } peers { permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 } } peers { permanent_uuid: "8e99d570bc764bb6b921a99d7ef151d1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40861 } }
I20260808 17:02:39.595613 16925 leader_election.cc:288] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df028094f95f442cb34bf7ff4c6bfe86 (127.30.194.3:39087), b6d3e67f735848ec800f3d75e8847740 (127.30.194.2:44899)
I20260808 17:02:39.599500 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.595903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:45985 (local address 127.30.194.3:39087)
0808 17:02:39.595964 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:39.595967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:39.596061 (+ 94us) server_negotiation.cc:407] Connection header received
0808 17:02:39.596098 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:39.596099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:39.596108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:39.596180 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:39.596356 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.597136 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.598587 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.599137 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.599214 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.599223 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:39.599396 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 17:02:39.599430 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 17:02:39.599500 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.595978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:56165 (local address 127.30.194.2:44899)
0808 17:02:39.596018 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:39.596020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:39.596035 (+ 15us) server_negotiation.cc:407] Connection header received
0808 17:02:39.596109 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:39.596110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:39.596119 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:39.596180 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:02:39.596357 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.597129 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.598587 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.599137 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.599213 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.599223 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:39.599394 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 17:02:39.599430 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:02:39.599534 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.595824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:44899 (local address 127.30.194.1:56165)
0808 17:02:39.595979 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:39.595989 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:39.596047 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:39.596191 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:39.596192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:39.596195 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:39.596290 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.596294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.597200 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.597201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.598505 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.598508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.599226 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.599228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.599317 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.599322 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:02:39.599381 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:02:39.599429 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":62,"spinlock_wait_cycles":15616,"thread_start_us":70,"threads_started":1}
I20260808 17:02:39.599534 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.595824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:39087 (local address 127.30.194.1:45985)
0808 17:02:39.595977 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:39.595989 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:02:39.596050 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:02:39.596190 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:02:39.596191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:02:39.596193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:02:39.596291 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.596293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.597202 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.597202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.598505 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:02:39.598508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.599227 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.599228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:02:39.599317 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.599323 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:02:39.599381 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:02:39.599429 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":65,"thread_start_us":79,"threads_started":1}
I20260808 17:02:39.599927 16631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0762fc43b746b9aec57127ee97d5d0" candidate_uuid: "8e99d570bc764bb6b921a99d7ef151d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6d3e67f735848ec800f3d75e8847740" is_pre_election: true
I20260808 17:02:39.600010 16829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0762fc43b746b9aec57127ee97d5d0" candidate_uuid: "8e99d570bc764bb6b921a99d7ef151d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df028094f95f442cb34bf7ff4c6bfe86" is_pre_election: true
I20260808 17:02:39.600076 16631 raft_consensus.cc:2454] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e99d570bc764bb6b921a99d7ef151d1 in term 0.
I20260808 17:02:39.600164 16829 raft_consensus.cc:2454] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e99d570bc764bb6b921a99d7ef151d1 in term 0.
I20260808 17:02:39.600315 16325 leader_election.cc:302] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e99d570bc764bb6b921a99d7ef151d1, b6d3e67f735848ec800f3d75e8847740; no voters:
I20260808 17:02:39.600445 16925 raft_consensus.cc:2781] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:02:39.600512 16925 raft_consensus.cc:482] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:02:39.600538 16925 raft_consensus.cc:3037] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:39.601281 16925 raft_consensus.cc:504] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 } } peers { permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 } } peers { permanent_uuid: "8e99d570bc764bb6b921a99d7ef151d1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40861 } }
I20260808 17:02:39.601401 16925 leader_election.cc:288] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [CANDIDATE]: Term 1 election: Requested vote from peers df028094f95f442cb34bf7ff4c6bfe86 (127.30.194.3:39087), b6d3e67f735848ec800f3d75e8847740 (127.30.194.2:44899)
I20260808 17:02:39.601560 16631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0762fc43b746b9aec57127ee97d5d0" candidate_uuid: "8e99d570bc764bb6b921a99d7ef151d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6d3e67f735848ec800f3d75e8847740"
I20260808 17:02:39.601608 16829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0762fc43b746b9aec57127ee97d5d0" candidate_uuid: "8e99d570bc764bb6b921a99d7ef151d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df028094f95f442cb34bf7ff4c6bfe86"
I20260808 17:02:39.601631 16631 raft_consensus.cc:3037] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:39.601687 16829 raft_consensus.cc:3037] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:02:39.602308 16631 raft_consensus.cc:2454] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e99d570bc764bb6b921a99d7ef151d1 in term 1.
I20260808 17:02:39.602365 16829 raft_consensus.cc:2454] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e99d570bc764bb6b921a99d7ef151d1 in term 1.
I20260808 17:02:39.602484 16325 leader_election.cc:302] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e99d570bc764bb6b921a99d7ef151d1, b6d3e67f735848ec800f3d75e8847740; no voters:
I20260808 17:02:39.602599 16925 raft_consensus.cc:2781] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:02:39.602749 16925 raft_consensus.cc:686] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 1 LEADER]: Becoming Leader. State: Replica: 8e99d570bc764bb6b921a99d7ef151d1, State: Running, Role: LEADER
I20260808 17:02:39.602846 16925 consensus_queue.cc:237] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 } } peers { permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 } } peers { permanent_uuid: "8e99d570bc764bb6b921a99d7ef151d1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40861 } }
I20260808 17:02:39.603642 16187 catalog_manager.cc:5495] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e99d570bc764bb6b921a99d7ef151d1 (127.30.194.1). New cstate: current_term: 1 leader_uuid: "8e99d570bc764bb6b921a99d7ef151d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e99d570bc764bb6b921a99d7ef151d1" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40861 } health_report { overall_health: HEALTHY } } }
I20260808 17:02:39.647444 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.645182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.30.194.1:40861)
0808 17:02:39.645235 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:02:39.645237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:02:39.645258 (+ 21us) server_negotiation.cc:407] Connection header received
0808 17:02:39.645344 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:02:39.645345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:02:39.645363 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:02:39.645436 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:02:39.645570 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.646033 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.646753 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:02:39.646935 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:02:39.646990 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:02:39.647149 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:02:39.647243 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:02:39.647311 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:02:39.647345 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 17:02:39.647364 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:02:39.649161 16829 raft_consensus.cc:1261] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86 [term 1 FOLLOWER]: Refusing update from remote peer 8e99d570bc764bb6b921a99d7ef151d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:39.649161 16631 raft_consensus.cc:1261] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740 [term 1 FOLLOWER]: Refusing update from remote peer 8e99d570bc764bb6b921a99d7ef151d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:02:39.649490 16933 consensus_queue.cc:1035] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6d3e67f735848ec800f3d75e8847740" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:39.649610 16925 consensus_queue.cc:1035] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df028094f95f442cb34bf7ff4c6bfe86" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:02:39.651738 16936 mvcc.cc:205] Tried to move back new op lower bound from 7316310260320292864 to 7316310260133756928. Current Snapshot: MvccSnapshot[applied={T|T < 7316310260320292864}]
I20260808 17:02:39.651932 16938 mvcc.cc:205] Tried to move back new op lower bound from 7316310260320292864 to 7316310260133756928. Current Snapshot: MvccSnapshot[applied={T|T < 7316310260320292864}]
I20260808 17:02:39.653913 16937 mvcc.cc:205] Tried to move back new op lower bound from 7316310260320292864 to 7316310260133756928. Current Snapshot: MvccSnapshot[applied={T|T < 7316310260320292864 or (T in {7316310260320292864})}]
I20260808 17:02:39.654175 16187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:39.654259 16187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:02:39.654898 16187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27cee1970aff4b44aa94d71105aa51a2: Sending DeleteTablet for 3 replicas of tablet 0e0762fc43b746b9aec57127ee97d5d0
I20260808 17:02:39.655221 16797 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0762fc43b746b9aec57127ee97d5d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:39 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:59312
I20260808 17:02:39.655211 16391 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0762fc43b746b9aec57127ee97d5d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:39 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:56012
I20260808 17:02:39.655431 16951 tablet_replica.cc:330] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: stopping tablet replica
I20260808 17:02:39.655428 16598 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0762fc43b746b9aec57127ee97d5d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:02:39 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:37332
I20260808 17:02:39.655527 16951 raft_consensus.cc:2229] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:39.655620 16951 raft_consensus.cc:2258] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:39.655689 16954 tablet_replica.cc:330] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: stopping tablet replica
I20260808 17:02:39.655894 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 16295
I20260808 17:02:39.655853 16953 tablet_replica.cc:330] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1: stopping tablet replica
I20260808 17:02:39.655990 16953 raft_consensus.cc:2229] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:02:39.656072 16953 raft_consensus.cc:2258] T 0e0762fc43b746b9aec57127ee97d5d0 P 8e99d570bc764bb6b921a99d7ef151d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:39.656247 16951 ts_tablet_manager.cc:1905] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:39.657580 16954 raft_consensus.cc:2229] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:02:39.657727 16954 raft_consensus.cc:2258] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:02:39.657876 16951 ts_tablet_manager.cc:1918] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:39.657927 16951 log.cc:1196] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/0e0762fc43b746b9aec57127ee97d5d0
I20260808 17:02:39.658152 16951 ts_tablet_manager.cc:1939] T 0e0762fc43b746b9aec57127ee97d5d0 P df028094f95f442cb34bf7ff4c6bfe86: Deleting consensus metadata
I20260808 17:02:39.660938 16954 ts_tablet_manager.cc:1905] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:02:39.662204 16954 ts_tablet_manager.cc:1918] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:02:39.662250 16954 log.cc:1196] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/0e0762fc43b746b9aec57127ee97d5d0
I20260808 17:02:39.662413 16954 ts_tablet_manager.cc:1939] T 0e0762fc43b746b9aec57127ee97d5d0 P b6d3e67f735848ec800f3d75e8847740: Deleting consensus metadata
I20260808 17:02:39.662791 16168 catalog_manager.cc:4841] TS b6d3e67f735848ec800f3d75e8847740 (127.30.194.2:44899): tablet 0e0762fc43b746b9aec57127ee97d5d0 (table test-table [id=dd22a429b3944576833cac472563d41f]) successfully deleted
I20260808 17:02:39.662791 16167 catalog_manager.cc:4841] TS df028094f95f442cb34bf7ff4c6bfe86 (127.30.194.3:39087): tablet 0e0762fc43b746b9aec57127ee97d5d0 (table test-table [id=dd22a429b3944576833cac472563d41f]) successfully deleted
W20260808 17:02:39.667397 16169 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:40861 (error 108)
I20260808 17:02:39.667821 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:02:39.667702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:40861 (local address 127.0.0.1:56026)
0808 17:02:39.667753 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 17:02:39.667791 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:40861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 17:02:39.667897 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 16506
W20260808 17:02:39.667922 16169 catalog_manager.cc:4584] TS 8e99d570bc764bb6b921a99d7ef151d1 (127.30.194.1:40861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e0762fc43b746b9aec57127ee97d5d0: Network error: Client connection negotiation failed: client connection to 127.30.194.1:40861: connect: Connection refused (error 111)
I20260808 17:02:39.678427 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 16708
I20260808 17:02:39.693442 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 16137
2026-08-08T17:02:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2372 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:02:39 dist-test-slave-2j7r krb5kdc[16958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:02:39 dist-test-slave-2j7r krb5kdc[16958](info): set up 2 sockets
Aug 08 17:02:39 dist-test-slave-2j7r krb5kdc[16958](info): commencing operation
krb5kdc: starting...
W20260808 17:02:40.838248 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:02:40 dist-test-slave-2j7r krb5kdc[16958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:02:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:02:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/rangeradmin_127.30.194.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/rangeradmin_127.30.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/rangerlookup_127.30.194.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/rangerlookup_127.30.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/HTTP_127.30.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/HTTP_127.30.194.20@KRBTEST.COM.keytab.
I20260808 17:02:40.983451 31496 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5D6OT5/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/postgres -l logfile start
2026-08-08 17:02:43.157 UTC [17007] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 17:02:43.157 UTC [17007] LOG: listening on IPv4 address "127.30.194.20", port 34679
2026-08-08 17:02:43.160 UTC [17007] LOG: listening on Unix socket "/tmp/.s.PGSQL.34679"
2026-08-08 17:02:43.187 UTC [17011] LOG: database system was shut down at 2026-08-08 17:02:41 UTC
2026-08-08 17:02:43.190 UTC [17007] LOG: database system is ready to accept connections
I20260808 17:02:44.257925 31496 mini_postgres.cc:96] Postgres bound to 34679
2026-08-08 17:02:44.264 UTC [17022] FATAL: database "slave" does not exist
127.30.194.20:34679 - accepting connections
I20260808 17:02:44.265545 31496 mini_ranger.cc:161] Starting Ranger...
I20260808 17:02:44.279846 31496 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 17:02:44.677109 31496 mini_ranger.cc:87] Created ranger Postgres database
I20260808 17:02:44.677182 31496 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-admin
2026-08-08 17:02:45,036 [I] DB FLAVOR :POSTGRES
2026-08-08 17:02:45,036 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 17:02:45,037 [I] Checking connection..
2026-08-08 17:02:45,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 17:02:45,311 [I] Checking connection passed.
2026-08-08 17:02:45,311 [I] --------- Verifying version history table ---------
2026-08-08 17:02:45,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:02:45,552 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 17:02:45,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:02:45,794 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 17:02:45,795 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 17:02:45,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 17:02:46.040 UTC [17125] WARNING: there is no transaction in progress
2026-08-08 17:02:46,048 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 17:02:46,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:02:46,299 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 17:02:46,299 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 17:02:46,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 17:02:46,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 17:02:46,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2j7r.c.gcp-upstream.internal','N') ;"
2026-08-08 17:02:47,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 17:02:47,300 [I] Table x_portal_user does not exist in database ranger
2026-08-08 17:02:47,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 17:02:47,536 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 17:02:47,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 17:02:47,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 17:02:48,026 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 17:02:48,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 17:02:48.886 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:48.901 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:48.915 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:48.928 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:48.943 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:48.960 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49.009 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49.015 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49.027 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49.036 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49.045 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49.058 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49.067 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49.073 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49.079 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49.085 UTC [17303] WARNING: there is no transaction in progress
2026-08-08 17:02:49,640 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 17:02:49,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 17:02:49,864 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 17:02:49,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 17:02:50,122 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 17:02:50,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 17:02:50,363 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 17:02:50,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 17:02:50,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 17:02:50,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2j7r.c.gcp-upstream.internal';"
2026-08-08 17:02:51,090 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 17:02:51,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 17:02:51,337 [I] DB_PATCHES have already been applied
I20260808 17:02:51.343204 31496 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-admin:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5D6OT5/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 17:02:51.343277 31496 mini_ranger.cc:193] Using host: 127.30.194.20
I20260808 17:02:51.346875 31496 mini_ranger.cc:227] Ranger admin URL: http://127.30.194.20:34005
Aug 08, 2026 5:02:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 5:02:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 5:02:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 5:02:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 5:02:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 5:02:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 5:02:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 17:02:52.601852 31496 mini_ranger.cc:160] Time spent starting Ranger: real 8.336s user 0.005s sys 0.005s
Aug 08, 2026 5:03:10 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 5:03:10 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 5:03:10 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 5:03:10 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 5:03:11 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 17:03:12.265231 31496 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/rangerkms_127.30.194.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/rangerkms_127.30.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.194.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.194.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.194.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/HTTP_127.30.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.194.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/HTTP_127.30.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 17:03:12 dist-test-slave-2j7r krb5kdc[16958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208592, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 17:03:12.340565 31496 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 17:03:12.349148 31496 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 17:03:13.144986 31496 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 17:03:13.145123 31496 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-kms
I20260808 17:03:13.145166 31496 mini_ranger_kms.cc:227] Using postgres at 127.30.194.20:34679
2026-08-08 17:03:13,586 [I] DB FLAVOR :POSTGRES
2026-08-08 17:03:13,586 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 17:03:13,587 [I] Checking connection
2026-08-08 17:03:13,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 17:03:13,949 [I] connection success
2026-08-08 17:03:13,949 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 17:03:13,950 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 17:03:13,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 17:03:14,359 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 17:03:14,360 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 17:03:14,360 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 17:03:14,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:34679/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 17:03:14,818 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 17:03:15.570539 31496 mini_ranger_kms.cc:310] Created kms service
I20260808 17:03:15.738854 31496 mini_ranger_kms.cc:326] Created kudu user
I20260808 17:03:15.777675 31496 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 17:03:15 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208592, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:03:16.150602 31496 mini_ranger_kms.cc:384] Added ranger policy
I20260808 17:03:16.150729 31496 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-kms:/tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 17:03:16.150755 31496 mini_ranger_kms.cc:240] Using host: 127.30.194.20
I20260808 17:03:16.154894 31496 mini_ranger_kms.cc:276] Ranger KMS PID: 17613
I20260808 17:03:16.155016 31496 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.194.20:60687
Aug 08, 2026 5:03:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 5:03:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 5:03:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 5:03:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 5:03:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 5:03:17 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 17:03:18 dist-test-slave-2j7r krb5kdc[16958](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208598, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.194.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:03:18.626343 31496 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.286s user 0.005s sys 0.401s
I20260808 17:03:18.626451 31496 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 17:03:18.626529 31496 mini_ranger_kms.cc:401] 127.30.194.20:60687/kms/v1/keys
Aug 08 17:03:18 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208598, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.194.20@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
Aug 08 17:03:19 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208598, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.194.20@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
Aug 08 17:03:21 dist-test-slave-2j7r krb5kdc[16958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:03:21.469379 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:36093
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:36093
--ranger_config_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:60687/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:03:21.579519 17698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:03:21.579717 17698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:03:21.579747 17698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:03:21.580842 17698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:03:21.580881 17698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:03:21.580900 17698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:03:21.580916 17698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:03:21.580932 17698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:03:21.582482 17698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.194.20:60687/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:36093
--ranger_config_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:36093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:03:21.582701 17698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:03:21.582890 17698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:03:21.587741 17708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:03:21.587742 17709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:03:21.587982 17711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:03:21.587868 17698 server_base.cc:1034] running on GCE node
I20260808 17:03:21.588363 17698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:03:21.588703 17698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:03:21.589872 17698 hybrid_clock.cc:648] HybridClock initialized: now 1786208601589853 us; error 32 us; skew 500 ppm
Aug 08 17:03:21 dist-test-slave-2j7r krb5kdc[16958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208601, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:03:21.596978 17698 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:03:21.597441 17698 webserver.cc:460] Webserver started at http://127.30.194.62:44141/ using document root <none> and password file <none>
I20260808 17:03:21.597626 17698 fs_manager.cc:359] Metadata directory not provided
I20260808 17:03:21.597666 17698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:03:21.597780 17698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:03:21 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208601, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:03:21.656651 17698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "7789a4010ebe4e338aaf75a0131fc3eb"
format_stamp: "Formatted at 2026-08-08 17:03:21 on dist-test-slave-2j7r"
server_key: "1898f02cbed1effab730864830f600af"
server_key_iv: "da8dd6dcb1e73747879dd4021188e61e"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:21.656980 17698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "7789a4010ebe4e338aaf75a0131fc3eb"
format_stamp: "Formatted at 2026-08-08 17:03:21 on dist-test-slave-2j7r"
server_key: "1898f02cbed1effab730864830f600af"
server_key_iv: "da8dd6dcb1e73747879dd4021188e61e"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:21.658497 17698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:03:21.672277 17724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:03:21.672488 17698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:03:21.672602 17698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "7789a4010ebe4e338aaf75a0131fc3eb"
format_stamp: "Formatted at 2026-08-08 17:03:21 on dist-test-slave-2j7r"
server_key: "1898f02cbed1effab730864830f600af"
server_key_iv: "da8dd6dcb1e73747879dd4021188e61e"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:21.672684 17698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:03:21.687032 17698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:03:21.688970 17698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:03:21.689155 17698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:03:21.694555 17698 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:36093
I20260808 17:03:21.694586 17800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:36093 every 8 connection(s)
I20260808 17:03:21.695048 17698 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:03:21.695350 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 17698
I20260808 17:03:21.695454 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:03:21.695664 31496 external_mini_cluster.cc:1428] Setting key 32b2da0694fbc5d09d1aac621adc2a85
I20260808 17:03:21.696091 17803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:03:21.698673 17803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb: Bootstrap starting.
I20260808 17:03:21.699344 17803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb: Neither blocks nor log segments found. Creating new log.
I20260808 17:03:21.699648 17803 log.cc:824] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb: Log is configured to *not* fsync() on all Append() calls
I20260808 17:03:21.700418 17803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb: No bootstrap required, opened a new log
I20260808 17:03:21.702575 17803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7789a4010ebe4e338aaf75a0131fc3eb" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36093 } }
I20260808 17:03:21.702728 17803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:03:21.702766 17803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7789a4010ebe4e338aaf75a0131fc3eb, State: Initialized, Role: FOLLOWER
I20260808 17:03:21.702917 17803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7789a4010ebe4e338aaf75a0131fc3eb" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36093 } }
I20260808 17:03:21.702994 17803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:03:21.703025 17803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:03:21.703086 17803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:03:21.703575 17803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7789a4010ebe4e338aaf75a0131fc3eb" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36093 } }
I20260808 17:03:21.703668 17803 leader_election.cc:302] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7789a4010ebe4e338aaf75a0131fc3eb; no voters:
Aug 08 17:03:21 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:03:21.704383 17803 leader_election.cc:288] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:03:21.704433 17810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:03:21.704600 17810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [term 1 LEADER]: Becoming Leader. State: Replica: 7789a4010ebe4e338aaf75a0131fc3eb, State: Running, Role: LEADER
I20260808 17:03:21.704749 17803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:03:21.706231 17810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7789a4010ebe4e338aaf75a0131fc3eb" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36093 } }
I20260808 17:03:21.707015 17803 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 17:03:21.707718 17813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7789a4010ebe4e338aaf75a0131fc3eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7789a4010ebe4e338aaf75a0131fc3eb" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36093 } } }
I20260808 17:03:21.707796 17813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [sys.catalog]: This master's current role is: LEADER
I20260808 17:03:21.708137 17811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7789a4010ebe4e338aaf75a0131fc3eb. Latest consensus state: current_term: 1 leader_uuid: "7789a4010ebe4e338aaf75a0131fc3eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7789a4010ebe4e338aaf75a0131fc3eb" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36093 } } }
I20260808 17:03:21.708276 17811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb [sys.catalog]: This master's current role is: LEADER
I20260808 17:03:21.708496 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:21.696148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.30.194.62:36093)
0808 17:03:21.696329 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:21.696339 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:21.697319 (+ 980us) server_negotiation.cc:407] Connection header received
0808 17:03:21.697862 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:21.697871 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:21.697931 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:21.698079 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:03:21.698983 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:21.700173 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:21.701315 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:21.701961 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:21.702070 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:21.704751 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:03:21.704755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:03:21.704755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:03:21.704767 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:03:21.706682 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:03:21.707160 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:03:21.707164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:03:21.707164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:03:21.707259 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:03:21.707549 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:03:21.707550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:03:21.707551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:03:21.707856 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:03:21.708010 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:21.708152 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 17:03:21.708270 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260808 17:03:21.708606 17838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:03:21.711790 17838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:03:21.714113 17838 catalog_manager.cc:1347] Generated new cluster ID: ca4977ab75a34f60a996de114271ebc1
I20260808 17:03:21.714170 17838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:03:21.738783 17838 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:03:21.739619 17838 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:03:21.749305 17838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb: Generated new TSK 0
I20260808 17:03:21.749538 17838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:03:22.471424 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:60687/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36093
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:03:22.584952 17876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:03:22.585179 17876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:03:22.585224 17876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:03:22.586361 17876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:03:22.586406 17876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:03:22.586444 17876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:03:22.587894 17876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.194.20:60687/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:03:22.588117 17876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:03:22.588348 17876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:03:22.593304 17886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:03:22.593911 17887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:03:22.594201 17889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:03:22.594551 17876 server_base.cc:1034] running on GCE node
I20260808 17:03:22.594775 17876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:03:22.595026 17876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:03:22.598304 17876 hybrid_clock.cc:648] HybridClock initialized: now 1786208602597765 us; error 552 us; skew 500 ppm
Aug 08 17:03:22 dist-test-slave-2j7r krb5kdc[16958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208602, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:03:22.601498 17876 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:03:22.602059 17876 webserver.cc:460] Webserver started at http://127.30.194.1:40429/ using document root <none> and password file <none>
I20260808 17:03:22.602319 17876 fs_manager.cc:359] Metadata directory not provided
I20260808 17:03:22.602378 17876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:03:22.602506 17876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:03:22 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208602, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:03:22.617357 17876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "76a1f2ea100e416fb10c0bf6e24addab"
format_stamp: "Formatted at 2026-08-08 17:03:22 on dist-test-slave-2j7r"
server_key: "adb0315ac3bb4246802015441188a912"
server_key_iv: "8791946fae738d4ff68f83312f368793"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:22.617782 17876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "76a1f2ea100e416fb10c0bf6e24addab"
format_stamp: "Formatted at 2026-08-08 17:03:22 on dist-test-slave-2j7r"
server_key: "adb0315ac3bb4246802015441188a912"
server_key_iv: "8791946fae738d4ff68f83312f368793"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:22.620536 17876 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 17:03:22.637202 17901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:03:22.637355 17876 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260808 17:03:22.637447 17876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "76a1f2ea100e416fb10c0bf6e24addab"
format_stamp: "Formatted at 2026-08-08 17:03:22 on dist-test-slave-2j7r"
server_key: "adb0315ac3bb4246802015441188a912"
server_key_iv: "8791946fae738d4ff68f83312f368793"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:22.637528 17876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:03:22.654680 17876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:03:22.656471 17876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:03:22.656597 17876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:03:22.656783 17876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:03:22.657153 17876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:03:22.657197 17876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:03:22.657267 17876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:03:22.657310 17876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:03:22.664193 17876 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:40107
I20260808 17:03:22.664291 18087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:40107 every 8 connection(s)
I20260808 17:03:22.664556 17876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:03:22 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208602, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:03:22.672325 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 17876
I20260808 17:03:22.672441 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:03:22.672643 31496 external_mini_cluster.cc:1428] Setting key 879a1b70e991686caa0a3f6e3ba28338
I20260808 17:03:22.675385 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:22.665250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36093 (local address 127.30.194.1:34571)
0808 17:03:22.665442 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 17:03:22.665491 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 17:03:22.666177 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:03:22.666613 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:03:22.666616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:03:22.666796 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:03:22.666975 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:22.666979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:22.667990 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:22.667991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:22.668912 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:22.668915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:22.669475 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:22.669476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:22.669621 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:22.670144 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:03:22.670154 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:03:22.671786 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:03:22.673783 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:03:22.673790 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:03:22.673792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:03:22.674065 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:03:22.674303 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:03:22.674304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:03:22.674304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:03:22.674447 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:03:22.674740 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:03:22.674747 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:03:22.674914 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 17:03:22.675005 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 17:03:22.675150 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260808 17:03:22.676023 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:22.665312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:34571 (local address 127.30.194.62:36093)
0808 17:03:22.666383 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:22.666385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:22.666394 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:03:22.666423 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:22.666424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:22.666439 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:22.666526 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:03:22.667084 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:22.667897 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:22.669015 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:22.669379 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:22.669449 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:22.671895 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:03:22.671896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:03:22.671897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:03:22.671904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:03:22.673685 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:03:22.674153 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:03:22.674154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:03:22.674154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:03:22.674228 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:03:22.674526 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:03:22.674527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:03:22.674527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:03:22.674660 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:03:22.675766 (+ 1106us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:22.675818 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 17:03:22.675912 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":83,"threads_started":1}
I20260808 17:03:22.676726 18088 heartbeater.cc:347] Connected to a master server at 127.30.194.62:36093
I20260808 17:03:22.676824 18088 heartbeater.cc:464] Registering TS with master...
I20260808 17:03:22.677002 18088 heartbeater.cc:511] Master 127.30.194.62:36093 requested a full tablet report, sending...
I20260808 17:03:22.677667 17750 ts_manager.cc:194] Registered new tserver with Master: 76a1f2ea100e416fb10c0bf6e24addab (127.30.194.1:40107)
I20260808 17:03:22.678532 17750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:34571
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:03:22.705757 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:60687/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36093
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 17:03:22 dist-test-slave-2j7r krb5kdc[16958](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208602, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 17:03:22.825122 18102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:03:22.825393 18102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:03:22.825426 18102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:03:22.827118 18102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:03:22.827200 18102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:03:22.827271 18102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:03:22.829344 18102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.194.20:60687/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:03:22.829661 18102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:03:22.829879 18102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:03:22.834831 18117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:03:22.834849 18111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:03:22.835175 18112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:03:22.835291 18102 server_base.cc:1034] running on GCE node
I20260808 17:03:22.835525 18102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:03:22.835779 18102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:03:22.839123 18102 hybrid_clock.cc:648] HybridClock initialized: now 1786208602839096 us; error 35 us; skew 500 ppm
Aug 08 17:03:22 dist-test-slave-2j7r krb5kdc[16958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208602, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:03:22.841789 18102 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:03:22.842283 18102 webserver.cc:460] Webserver started at http://127.30.194.2:44417/ using document root <none> and password file <none>
I20260808 17:03:22.842530 18102 fs_manager.cc:359] Metadata directory not provided
I20260808 17:03:22.842583 18102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:03:22.842703 18102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:03:22 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208602, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:03:22.856092 18102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "7b585299c24b4cbe902eec33ff2a0454"
format_stamp: "Formatted at 2026-08-08 17:03:22 on dist-test-slave-2j7r"
server_key: "f117f8cd2b51f84363036b933dec1ad6"
server_key_iv: "6d7da4b5a148c2f09c145bbc529148eb"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:22.856582 18102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "7b585299c24b4cbe902eec33ff2a0454"
format_stamp: "Formatted at 2026-08-08 17:03:22 on dist-test-slave-2j7r"
server_key: "f117f8cd2b51f84363036b933dec1ad6"
server_key_iv: "6d7da4b5a148c2f09c145bbc529148eb"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:22.865509 18102 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.004s
I20260808 17:03:22.880780 18130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:03:22.881052 18102 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260808 17:03:22.881243 18102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "7b585299c24b4cbe902eec33ff2a0454"
format_stamp: "Formatted at 2026-08-08 17:03:22 on dist-test-slave-2j7r"
server_key: "f117f8cd2b51f84363036b933dec1ad6"
server_key_iv: "6d7da4b5a148c2f09c145bbc529148eb"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:22.881347 18102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:03:22.904098 18102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:03:22.906672 18102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:03:22.906883 18102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:03:22.907179 18102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:03:22.907589 18102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:03:22.907670 18102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:03:22.907739 18102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:03:22.907805 18102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:03:22.918918 18102 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:32999
I20260808 17:03:22.919387 18102 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:03:22.922343 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 18102
I20260808 17:03:22.922458 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:03:22.922668 31496 external_mini_cluster.cc:1428] Setting key db3dd2e7017bd269492941b917c630fc
I20260808 17:03:22.922237 18312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:32999 every 8 connection(s)
Aug 08 17:03:22 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208602, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
I20260808 17:03:22.942274 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:22.923204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:39865 (local address 127.30.194.62:36093)
0808 17:03:22.923264 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:22.923267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:22.929430 (+ 6163us) server_negotiation.cc:407] Connection header received
0808 17:03:22.931517 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:22.931519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:22.931535 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:22.931565 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:03:22.932384 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:22.933208 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:22.934682 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:22.935007 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:22.935047 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:22.938987 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:03:22.938988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:03:22.938989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:03:22.938996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:03:22.940667 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:03:22.941162 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:03:22.941164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:03:22.941164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:03:22.941225 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:03:22.941535 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:03:22.941537 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:03:22.941537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:03:22.941678 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:03:22.941724 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:22.942088 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 17:03:22.942177 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:03:22.942399 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:22.922975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36093 (local address 127.30.194.2:39865)
0808 17:03:22.929226 (+ 6251us) negotiation.cc:107] Waiting for socket to connect
0808 17:03:22.929274 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 17:03:22.930021 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:03:22.931769 (+ 1748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:03:22.931773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:03:22.932013 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:03:22.932278 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:22.932284 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:22.933321 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:22.933322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:22.934584 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:22.934586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:22.935134 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:22.935135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:22.935273 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:22.937245 (+ 1972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:03:22.937256 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:03:22.938860 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:03:22.940765 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:03:22.940772 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:03:22.940774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:03:22.941069 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:03:22.941306 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:03:22.941307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:03:22.941308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:03:22.941446 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:03:22.941803 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:03:22.941811 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:03:22.941943 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 17:03:22.942029 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 17:03:22.942181 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6201,"thread_start_us":84,"threads_started":1}
I20260808 17:03:22.942893 18313 heartbeater.cc:347] Connected to a master server at 127.30.194.62:36093
I20260808 17:03:22.942991 18313 heartbeater.cc:464] Registering TS with master...
I20260808 17:03:22.943186 18313 heartbeater.cc:511] Master 127.30.194.62:36093 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
I20260808 17:03:22.943560 17750 ts_manager.cc:194] Registered new tserver with Master: 7b585299c24b4cbe902eec33ff2a0454 (127.30.194.2:32999)
I20260808 17:03:22.943917 17750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:39865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:03:22.957885 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:60687/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36093
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:03:23.051141 18319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:03:23.051339 18319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:03:23.051378 18319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:03:23.052506 18319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:03:23.052548 18319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:03:23.052582 18319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:03:23.054092 18319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.194.20:60687/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:03:23.054334 18319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:03:23.054554 18319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:03:23.059321 18333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:03:23.059306 18336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:03:23.059443 18339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:03:23.059620 18319 server_base.cc:1034] running on GCE node
I20260808 17:03:23.059890 18319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:03:23.060168 18319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:03:23.061321 18319 hybrid_clock.cc:648] HybridClock initialized: now 1786208603061302 us; error 33 us; skew 500 ppm
Aug 08 17:03:23 dist-test-slave-2j7r krb5kdc[16958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208603, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:03:23.064356 18319 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:03:23.064910 18319 webserver.cc:460] Webserver started at http://127.30.194.3:46823/ using document root <none> and password file <none>
I20260808 17:03:23.065217 18319 fs_manager.cc:359] Metadata directory not provided
I20260808 17:03:23.065291 18319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:03:23.065425 18319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:03:23 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208603, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:03:23.077605 18319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "810c25c4d3d0417b8f01cdcf5e278a93"
format_stamp: "Formatted at 2026-08-08 17:03:23 on dist-test-slave-2j7r"
server_key: "1e1e985307b85d51bbaa7057ca01d612"
server_key_iv: "bc4e369a5b05c01c8214e1e688df6941"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:23.078277 18319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "810c25c4d3d0417b8f01cdcf5e278a93"
format_stamp: "Formatted at 2026-08-08 17:03:23 on dist-test-slave-2j7r"
server_key: "1e1e985307b85d51bbaa7057ca01d612"
server_key_iv: "bc4e369a5b05c01c8214e1e688df6941"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:23.080063 18319 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 17:03:23.097072 18348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:03:23.097321 18319 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260808 17:03:23.097417 18319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "810c25c4d3d0417b8f01cdcf5e278a93"
format_stamp: "Formatted at 2026-08-08 17:03:23 on dist-test-slave-2j7r"
server_key: "1e1e985307b85d51bbaa7057ca01d612"
server_key_iv: "bc4e369a5b05c01c8214e1e688df6941"
server_key_version: "kuduclusterkey@0"
I20260808 17:03:23.097498 18319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:03:23.129338 18319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:03:23.131603 18319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:03:23.131768 18319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:03:23.131986 18319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:03:23.132436 18319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:03:23.132484 18319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:03:23.132515 18319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:03:23.132580 18319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:03:23.140389 18319 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:45537
I20260808 17:03:23.140816 18319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:03:23.141396 18528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:45537 every 8 connection(s)
I20260808 17:03:23.144474 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 18319
I20260808 17:03:23.144587 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:03:23.144748 31496 external_mini_cluster.cc:1428] Setting key 3434b2792d92777b91805a7de02bfc38
Aug 08 17:03:23 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208603, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:03:23.152782 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.142053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:57795 (local address 127.30.194.62:36093)
0808 17:03:23.142111 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:23.142113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:23.142124 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:03:23.142640 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:23.142642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:23.142655 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:23.142721 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:03:23.143432 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.144241 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.146675 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.146987 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.147066 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.149569 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:03:23.149570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:03:23.149571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:03:23.149578 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:03:23.151221 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:03:23.151700 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:03:23.151702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:03:23.151702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:03:23.151760 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:03:23.152042 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:03:23.152043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:03:23.152043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:03:23.152175 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:03:23.152217 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:23.152597 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 17:03:23.152686 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:03:23.153352 18533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.141666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36093 (local address 127.30.194.3:57795)
0808 17:03:23.141863 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 17:03:23.141907 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:03:23.142523 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:03:23.142793 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:03:23.142797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:03:23.143039 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:03:23.143320 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.143326 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.145023 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.145023 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.146580 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.146583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.147091 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.147092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.147242 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.147825 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:03:23.147835 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:03:23.149455 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:03:23.151318 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:03:23.151324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:03:23.151326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:03:23.151613 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:03:23.151828 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:03:23.151829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:03:23.151829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:03:23.151968 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:03:23.152263 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:03:23.152270 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:03:23.152432 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 17:03:23.152922 (+ 490us) client_negotiation.cc:240] Negotiation successful
0808 17:03:23.153118 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260808 17:03:23.153748 18530 heartbeater.cc:347] Connected to a master server at 127.30.194.62:36093
I20260808 17:03:23.153846 18530 heartbeater.cc:464] Registering TS with master...
I20260808 17:03:23.154054 18530 heartbeater.cc:511] Master 127.30.194.62:36093 requested a full tablet report, sending...
I20260808 17:03:23.154402 17750 ts_manager.cc:194] Registered new tserver with Master: 810c25c4d3d0417b8f01cdcf5e278a93 (127.30.194.3:45537)
I20260808 17:03:23.154893 17750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:57795
I20260808 17:03:23.156415 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:03:23.166059 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.158365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53772 (local address 127.30.194.62:36093)
0808 17:03:23.158410 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:23.158412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:23.158707 (+ 295us) server_negotiation.cc:407] Connection header received
0808 17:03:23.158737 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:23.158738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:23.158748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:23.158820 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:03:23.159516 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.160233 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.161287 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.161568 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.161612 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.163065 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:03:23.163067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:03:23.163067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:03:23.163074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:03:23.164646 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:03:23.165054 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:03:23.165056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:03:23.165056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:03:23.165142 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:03:23.165363 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:03:23.165364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:03:23.165364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:03:23.165565 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:03:23.165644 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:23.165878 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 17:03:23.165973 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:03:23.168232 17750 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:03:23.170573 17750 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:03:23.182509 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.173751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:32999 (local address 127.0.0.1:34876)
0808 17:03:23.174698 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0808 17:03:23.174742 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:03:23.174809 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:03:23.175102 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:03:23.175105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:03:23.175108 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:03:23.175172 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.175174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.179367 (+ 4193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.179368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.180411 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.180414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.182207 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.182209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.182292 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.182308 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 17:03:23.182380 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:03:23.182433 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":90,"threads_started":1}
I20260808 17:03:23.183235 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.173999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:45537 (local address 127.0.0.1:35310)
0808 17:03:23.175274 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0808 17:03:23.175282 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:03:23.175323 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:03:23.175563 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:03:23.175564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:03:23.175565 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:03:23.175615 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.175616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.176931 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.176932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.178299 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.178303 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.182939 (+ 4636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.182940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.183067 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.183071 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:03:23.183128 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 17:03:23.183165 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1259,"thread_start_us":69,"threads_started":1}
I20260808 17:03:23.183822 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.174015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.30.194.2:32999)
0808 17:03:23.174821 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:23.174826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:23.174838 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:03:23.174914 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:23.174916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:23.174932 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:23.175012 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:03:23.175622 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.179071 (+ 3449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.180534 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.181325 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.181657 (+ 332us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.181748 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:23.183687 (+ 1939us) server_negotiation.cc:299] Negotiation successful
0808 17:03:23.183729 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":93,"threads_started":1}
I20260808 17:03:23.184017 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.174326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35310 (local address 127.30.194.3:45537)
0808 17:03:23.174984 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:23.174989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:23.175376 (+ 387us) server_negotiation.cc:407] Connection header received
0808 17:03:23.175418 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:23.175421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:23.175435 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:23.175498 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:03:23.175674 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.176800 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.178421 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.179177 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.179246 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.179330 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:23.183896 (+ 4566us) server_negotiation.cc:299] Negotiation successful
0808 17:03:23.183938 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":76,"threads_started":1}
I20260808 17:03:23.184613 18212 tablet_service.cc:1467] Processing CreateTablet for tablet 467a540e7735472284bda868f5255f1d (DEFAULT_TABLE table=test-table [id=37144859443243069a71e73ea639e996]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:03:23.184665 18396 tablet_service.cc:1467] Processing CreateTablet for tablet 467a540e7735472284bda868f5255f1d (DEFAULT_TABLE table=test-table [id=37144859443243069a71e73ea639e996]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:03:23.185012 18212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467a540e7735472284bda868f5255f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:03:23.185067 18396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467a540e7735472284bda868f5255f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:03:23.190968 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.174188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:40107 (local address 127.0.0.1:60930)
0808 17:03:23.180542 (+ 6354us) negotiation.cc:107] Waiting for socket to connect
0808 17:03:23.180552 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:03:23.180598 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:03:23.180867 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:03:23.180868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:03:23.180882 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:03:23.180931 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.180932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.183485 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.183486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.189687 (+ 6201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.189691 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.190729 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.190730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.190794 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.190798 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:03:23.190849 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:03:23.190900 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6335,"thread_start_us":75,"threads_started":1}
I20260808 17:03:23.191313 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.174501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60930 (local address 127.30.194.1:40107)
0808 17:03:23.175055 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:23.175059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:23.180656 (+ 5597us) server_negotiation.cc:407] Connection header received
0808 17:03:23.180705 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:23.180708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:23.180725 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:23.180796 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:03:23.181013 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.182105 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.189790 (+ 7685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.190561 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.190633 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.190691 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:23.191186 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 17:03:23.191234 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":62,"threads_started":1}
I20260808 17:03:23.191933 18564 tablet_bootstrap.cc:492] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: Bootstrap starting.
I20260808 17:03:23.192166 17974 tablet_service.cc:1467] Processing CreateTablet for tablet 467a540e7735472284bda868f5255f1d (DEFAULT_TABLE table=test-table [id=37144859443243069a71e73ea639e996]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:03:23.192492 17974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467a540e7735472284bda868f5255f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:03:23.193224 18561 tablet_bootstrap.cc:492] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: Bootstrap starting.
I20260808 17:03:23.193389 18564 tablet_bootstrap.cc:654] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: Neither blocks nor log segments found. Creating new log.
I20260808 17:03:23.194373 18561 tablet_bootstrap.cc:654] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: Neither blocks nor log segments found. Creating new log.
I20260808 17:03:23.194754 18564 log.cc:824] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: Log is configured to *not* fsync() on all Append() calls
I20260808 17:03:23.195363 18561 log.cc:824] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: Log is configured to *not* fsync() on all Append() calls
I20260808 17:03:23.196280 18564 tablet_bootstrap.cc:492] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: No bootstrap required, opened a new log
I20260808 17:03:23.196413 18564 ts_tablet_manager.cc:1397] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260808 17:03:23.196406 18567 tablet_bootstrap.cc:492] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab: Bootstrap starting.
I20260808 17:03:23.198536 18564 raft_consensus.cc:348] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 } } peers { permanent_uuid: "76a1f2ea100e416fb10c0bf6e24addab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40107 } } peers { permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 } }
I20260808 17:03:23.198668 18567 tablet_bootstrap.cc:654] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab: Neither blocks nor log segments found. Creating new log.
I20260808 17:03:23.198803 18564 raft_consensus.cc:374] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:03:23.198908 18564 raft_consensus.cc:729] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b585299c24b4cbe902eec33ff2a0454, State: Initialized, Role: FOLLOWER
I20260808 17:03:23.199113 18564 consensus_queue.cc:260] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 } } peers { permanent_uuid: "76a1f2ea100e416fb10c0bf6e24addab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40107 } } peers { permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 } }
I20260808 17:03:23.199651 18564 ts_tablet_manager.cc:1428] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:03:23.198969 18567 log.cc:824] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab: Log is configured to *not* fsync() on all Append() calls
I20260808 17:03:23.199824 18313 heartbeater.cc:503] Master 127.30.194.62:36093 was elected leader, sending a full tablet report...
I20260808 17:03:23.200521 18567 tablet_bootstrap.cc:492] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab: No bootstrap required, opened a new log
I20260808 17:03:23.200587 18567 ts_tablet_manager.cc:1397] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 17:03:23.201148 18561 tablet_bootstrap.cc:492] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: No bootstrap required, opened a new log
I20260808 17:03:23.201267 18561 ts_tablet_manager.cc:1397] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 17:03:23.203095 18567 raft_consensus.cc:348] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 } } peers { permanent_uuid: "76a1f2ea100e416fb10c0bf6e24addab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40107 } } peers { permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 } }
I20260808 17:03:23.203284 18567 raft_consensus.cc:374] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:03:23.203325 18567 raft_consensus.cc:729] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76a1f2ea100e416fb10c0bf6e24addab, State: Initialized, Role: FOLLOWER
I20260808 17:03:23.203491 18561 raft_consensus.cc:348] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 } } peers { permanent_uuid: "76a1f2ea100e416fb10c0bf6e24addab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40107 } } peers { permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 } }
I20260808 17:03:23.203742 18561 raft_consensus.cc:374] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:03:23.203727 18567 consensus_queue.cc:260] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 } } peers { permanent_uuid: "76a1f2ea100e416fb10c0bf6e24addab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40107 } } peers { permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 } }
I20260808 17:03:23.203825 18561 raft_consensus.cc:729] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 810c25c4d3d0417b8f01cdcf5e278a93, State: Initialized, Role: FOLLOWER
I20260808 17:03:23.204010 18567 ts_tablet_manager.cc:1428] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:03:23.204089 18561 consensus_queue.cc:260] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 } } peers { permanent_uuid: "76a1f2ea100e416fb10c0bf6e24addab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40107 } } peers { permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 } }
I20260808 17:03:23.204162 18088 heartbeater.cc:503] Master 127.30.194.62:36093 was elected leader, sending a full tablet report...
I20260808 17:03:23.204398 18561 ts_tablet_manager.cc:1428] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 17:03:23.204547 18530 heartbeater.cc:503] Master 127.30.194.62:36093 was elected leader, sending a full tablet report...
Aug 08 17:03:23 dist-test-slave-2j7r krb5kdc[16958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208602, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:03:23.310030 18573 raft_consensus.cc:482] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:03:23.310195 18573 raft_consensus.cc:504] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 } } peers { permanent_uuid: "76a1f2ea100e416fb10c0bf6e24addab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40107 } } peers { permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 } }
I20260808 17:03:23.310570 18573 leader_election.cc:288] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b585299c24b4cbe902eec33ff2a0454 (127.30.194.2:32999), 810c25c4d3d0417b8f01cdcf5e278a93 (127.30.194.3:45537)
I20260808 17:03:23.314545 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.310845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:60373 (local address 127.30.194.2:32999)
0808 17:03:23.310890 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:23.310892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:23.311116 (+ 224us) server_negotiation.cc:407] Connection header received
0808 17:03:23.311212 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:23.311213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:23.311224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:23.311250 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:03:23.311514 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.312219 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.313682 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.314194 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.314238 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.314247 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:23.314440 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 17:03:23.314476 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:03:23.314729 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.311177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:57595 (local address 127.30.194.3:45537)
0808 17:03:23.311220 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:23.311223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:23.311335 (+ 112us) server_negotiation.cc:407] Connection header received
0808 17:03:23.311437 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:23.311439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:23.311448 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:23.311478 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:03:23.311689 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.312423 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.313799 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.314348 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.314398 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.314406 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:23.314622 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 17:03:23.314657 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:03:23.314981 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.311020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:45537 (local address 127.30.194.1:57595)
0808 17:03:23.311278 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 17:03:23.311287 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:03:23.311360 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:03:23.311542 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:03:23.311543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:03:23.311545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:03:23.311617 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.311618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.312549 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.312550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.313687 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.313691 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.314467 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.314467 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.314548 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.314551 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:03:23.314844 (+ 293us) client_negotiation.cc:240] Negotiation successful
0808 17:03:23.314899 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":64,"threads_started":1}
I20260808 17:03:23.315204 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.310777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:32999 (local address 127.30.194.1:60373)
0808 17:03:23.311046 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 17:03:23.311058 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:03:23.311143 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:03:23.311315 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:03:23.311316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:03:23.311318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:03:23.311445 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.311447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.312329 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.312329 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.313584 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:03:23.313588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.314302 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.314303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:03:23.314372 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.314376 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:03:23.314919 (+ 543us) client_negotiation.cc:240] Negotiation successful
0808 17:03:23.314960 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":44,"thread_start_us":127,"threads_started":1}
I20260808 17:03:23.315315 18457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467a540e7735472284bda868f5255f1d" candidate_uuid: "76a1f2ea100e416fb10c0bf6e24addab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" is_pre_election: true
I20260808 17:03:23.315478 18457 raft_consensus.cc:2454] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76a1f2ea100e416fb10c0bf6e24addab in term 0.
I20260808 17:03:23.315578 18240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467a540e7735472284bda868f5255f1d" candidate_uuid: "76a1f2ea100e416fb10c0bf6e24addab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b585299c24b4cbe902eec33ff2a0454" is_pre_election: true
I20260808 17:03:23.315719 18240 raft_consensus.cc:2454] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76a1f2ea100e416fb10c0bf6e24addab in term 0.
I20260808 17:03:23.315987 17906 leader_election.cc:302] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76a1f2ea100e416fb10c0bf6e24addab, 7b585299c24b4cbe902eec33ff2a0454; no voters:
I20260808 17:03:23.316162 18573 raft_consensus.cc:2781] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:03:23.316205 18573 raft_consensus.cc:482] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:03:23.316223 18573 raft_consensus.cc:3037] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:03:23.317118 18573 raft_consensus.cc:504] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 } } peers { permanent_uuid: "76a1f2ea100e416fb10c0bf6e24addab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40107 } } peers { permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 } }
I20260808 17:03:23.317243 18573 leader_election.cc:288] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [CANDIDATE]: Term 1 election: Requested vote from peers 7b585299c24b4cbe902eec33ff2a0454 (127.30.194.2:32999), 810c25c4d3d0417b8f01cdcf5e278a93 (127.30.194.3:45537)
I20260808 17:03:23.317462 18240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467a540e7735472284bda868f5255f1d" candidate_uuid: "76a1f2ea100e416fb10c0bf6e24addab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b585299c24b4cbe902eec33ff2a0454"
I20260808 17:03:23.317482 18457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467a540e7735472284bda868f5255f1d" candidate_uuid: "76a1f2ea100e416fb10c0bf6e24addab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "810c25c4d3d0417b8f01cdcf5e278a93"
I20260808 17:03:23.317538 18240 raft_consensus.cc:3037] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:03:23.317561 18457 raft_consensus.cc:3037] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:03:23.318365 18240 raft_consensus.cc:2454] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76a1f2ea100e416fb10c0bf6e24addab in term 1.
I20260808 17:03:23.318467 18457 raft_consensus.cc:2454] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76a1f2ea100e416fb10c0bf6e24addab in term 1.
I20260808 17:03:23.318529 17906 leader_election.cc:302] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76a1f2ea100e416fb10c0bf6e24addab, 7b585299c24b4cbe902eec33ff2a0454; no voters:
I20260808 17:03:23.318698 18573 raft_consensus.cc:2781] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:03:23.318861 18573 raft_consensus.cc:686] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [term 1 LEADER]: Becoming Leader. State: Replica: 76a1f2ea100e416fb10c0bf6e24addab, State: Running, Role: LEADER
I20260808 17:03:23.318957 18573 consensus_queue.cc:237] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 } } peers { permanent_uuid: "76a1f2ea100e416fb10c0bf6e24addab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40107 } } peers { permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 } }
I20260808 17:03:23.319778 17748 catalog_manager.cc:5495] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab reported cstate change: term changed from 0 to 1, leader changed from <none> to 76a1f2ea100e416fb10c0bf6e24addab (127.30.194.1). New cstate: current_term: 1 leader_uuid: "76a1f2ea100e416fb10c0bf6e24addab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76a1f2ea100e416fb10c0bf6e24addab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 40107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 } health_report { overall_health: UNKNOWN } } }
I20260808 17:03:23.331400 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.329149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60940 (local address 127.30.194.1:40107)
0808 17:03:23.329187 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:03:23.329189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:03:23.329201 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:03:23.329273 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:03:23.329273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:03:23.329288 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:03:23.329359 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:03:23.329502 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.329952 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.330668 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:03:23.330876 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:03:23.330928 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:03:23.331081 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:03:23.331184 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:03:23.331253 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:03:23.331291 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 17:03:23.331311 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:03:23.333074 18457 raft_consensus.cc:1261] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93 [term 1 FOLLOWER]: Refusing update from remote peer 76a1f2ea100e416fb10c0bf6e24addab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:03:23.333133 18240 raft_consensus.cc:1261] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454 [term 1 FOLLOWER]: Refusing update from remote peer 76a1f2ea100e416fb10c0bf6e24addab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:03:23.333441 18577 consensus_queue.cc:1035] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [LEADER]: Connected to new peer: Peer: permanent_uuid: "810c25c4d3d0417b8f01cdcf5e278a93" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:03:23.333660 18573 consensus_queue.cc:1035] T 467a540e7735472284bda868f5255f1d P 76a1f2ea100e416fb10c0bf6e24addab [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b585299c24b4cbe902eec33ff2a0454" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 32999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:03:23.336180 18581 mvcc.cc:205] Tried to move back new op lower bound from 7316310439249907712 to 7316310439195062272. Current Snapshot: MvccSnapshot[applied={T|T < 7316310439249907712}]
I20260808 17:03:23.337513 18585 mvcc.cc:205] Tried to move back new op lower bound from 7316310439249907712 to 7316310439195062272. Current Snapshot: MvccSnapshot[applied={T|T < 7316310439249907712}]
I20260808 17:03:23.338539 17748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:03:23.338626 17748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:03:23.338718 18583 mvcc.cc:205] Tried to move back new op lower bound from 7316310439249907712 to 7316310439195062272. Current Snapshot: MvccSnapshot[applied={T|T < 7316310439249907712}]
I20260808 17:03:23.339426 17748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7789a4010ebe4e338aaf75a0131fc3eb: Sending DeleteTablet for 3 replicas of tablet 467a540e7735472284bda868f5255f1d
I20260808 17:03:23.339710 17974 tablet_service.cc:1514] Processing DeleteTablet for tablet 467a540e7735472284bda868f5255f1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:03:23 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:60930
I20260808 17:03:23.339740 18396 tablet_service.cc:1514] Processing DeleteTablet for tablet 467a540e7735472284bda868f5255f1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:03:23 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:35310
I20260808 17:03:23.339725 17965 tablet_service.cc:1514] Processing DeleteTablet for tablet 467a540e7735472284bda868f5255f1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:03:23 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:60930
I20260808 17:03:23.340008 18427 tablet_service.cc:1514] Processing DeleteTablet for tablet 467a540e7735472284bda868f5255f1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:03:23 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:35310
I20260808 17:03:23.340310 18212 tablet_service.cc:1514] Processing DeleteTablet for tablet 467a540e7735472284bda868f5255f1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:03:23 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:34876
I20260808 17:03:23.340595 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 17876
I20260808 17:03:23.340026 18598 tablet_replica.cc:330] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: stopping tablet replica
I20260808 17:03:23.359982 18598 raft_consensus.cc:2229] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:03:23.360072 18598 raft_consensus.cc:2258] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:03:23.360220 18602 tablet_replica.cc:330] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: stopping tablet replica
I20260808 17:03:23.360306 18602 raft_consensus.cc:2229] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:03:23.360348 18602 raft_consensus.cc:2258] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:03:23.376019 18598 ts_tablet_manager.cc:1905] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:03:23.378033 18598 ts_tablet_manager.cc:1918] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:03:23.378105 18598 log.cc:1196] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/467a540e7735472284bda868f5255f1d
I20260808 17:03:23.378539 18598 ts_tablet_manager.cc:1939] T 467a540e7735472284bda868f5255f1d P 810c25c4d3d0417b8f01cdcf5e278a93: Deleting consensus metadata
I20260808 17:03:23.378985 17729 catalog_manager.cc:4841] TS 810c25c4d3d0417b8f01cdcf5e278a93 (127.30.194.3:45537): tablet 467a540e7735472284bda868f5255f1d (table test-table [id=37144859443243069a71e73ea639e996]) successfully deleted
W20260808 17:03:23.379150 17729 catalog_manager.cc:4808] TS 810c25c4d3d0417b8f01cdcf5e278a93 (127.30.194.3:45537): delete failed for tablet 467a540e7735472284bda868f5255f1d because the tablet was not found. No further retry: Not found: Tablet not found: 467a540e7735472284bda868f5255f1d
I20260808 17:03:23.379154 18602 ts_tablet_manager.cc:1905] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:03:23.380364 18602 ts_tablet_manager.cc:1918] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:03:23.380432 18602 log.cc:1196] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/467a540e7735472284bda868f5255f1d
I20260808 17:03:23.380646 18602 ts_tablet_manager.cc:1939] T 467a540e7735472284bda868f5255f1d P 7b585299c24b4cbe902eec33ff2a0454: Deleting consensus metadata
I20260808 17:03:23.381031 17729 catalog_manager.cc:4841] TS 7b585299c24b4cbe902eec33ff2a0454 (127.30.194.2:32999): tablet 467a540e7735472284bda868f5255f1d (table test-table [id=37144859443243069a71e73ea639e996]) successfully deleted
W20260808 17:03:23.388813 17733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:40107 (error 108)
I20260808 17:03:23.389038 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 18102
I20260808 17:03:23.391172 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:03:23.389222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:40107 (local address 127.0.0.1:60956)
0808 17:03:23.391083 (+ 1861us) negotiation.cc:107] Waiting for socket to connect
0808 17:03:23.391134 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:40107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1834}
W20260808 17:03:23.391386 17733 catalog_manager.cc:4584] TS 76a1f2ea100e416fb10c0bf6e24addab (127.30.194.1:40107): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 467a540e7735472284bda868f5255f1d: Network error: Client connection negotiation failed: client connection to 127.30.194.1:40107: connect: Connection refused (error 111)
I20260808 17:03:23.398268 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 18319
I20260808 17:03:23.420002 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 17698
I20260808 17:03:23.433368 31496 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 17:03:23.777222 31496 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 17:03:23.777338 31496 mini_ranger.cc:66] Stopping Ranger...
I20260808 17:03:24.120599 31496 mini_ranger.cc:68] Stopped Ranger
2026-08-08 17:03:24.120 UTC [17007] LOG: received smart shutdown request
2026-08-08 17:03:24.124 UTC [17007] LOG: background worker "logical replication launcher" (PID 17019) exited with exit code 1
2026-08-08 17:03:24.125 UTC [17012] LOG: shutting down
2026-08-08 17:03:24.269 UTC [17007] LOG: database system is shut down
2026-08-08T17:03:24Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44657 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:03:24 dist-test-slave-2j7r krb5kdc[18614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:03:24 dist-test-slave-2j7r krb5kdc[18614](info): set up 2 sockets
Aug 08 17:03:24 dist-test-slave-2j7r krb5kdc[18614](info): commencing operation
krb5kdc: starting...
W20260808 17:03:25.510242 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:03:25 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:03:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:03:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/rangeradmin_127.30.194.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/rangeradmin_127.30.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/rangerlookup_127.30.194.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/rangerlookup_127.30.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/HTTP_127.30.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/HTTP_127.30.194.20@KRBTEST.COM.keytab.
I20260808 17:03:25.663735 31496 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5D6OT5/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/postgres -l logfile start
2026-08-08 17:03:27.936 UTC [18676] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 17:03:27.936 UTC [18676] LOG: listening on IPv4 address "127.30.194.20", port 58067
2026-08-08 17:03:27.939 UTC [18676] LOG: listening on Unix socket "/tmp/.s.PGSQL.58067"
2026-08-08 17:03:27.966 UTC [18679] LOG: database system was shut down at 2026-08-08 17:03:26 UTC
2026-08-08 17:03:27.970 UTC [18676] LOG: database system is ready to accept connections
I20260808 17:03:29.046978 31496 mini_postgres.cc:96] Postgres bound to 58067
2026-08-08 17:03:29.053 UTC [18689] FATAL: database "slave" does not exist
127.30.194.20:58067 - accepting connections
I20260808 17:03:29.054257 31496 mini_ranger.cc:161] Starting Ranger...
I20260808 17:03:29.068929 31496 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 17:03:29.525353 31496 mini_ranger.cc:87] Created ranger Postgres database
I20260808 17:03:29.525432 31496 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-admin
2026-08-08 17:03:29,793 [I] DB FLAVOR :POSTGRES
2026-08-08 17:03:29,793 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 17:03:29,793 [I] Checking connection..
2026-08-08 17:03:29,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 17:03:30,042 [I] Checking connection passed.
2026-08-08 17:03:30,043 [I] --------- Verifying version history table ---------
2026-08-08 17:03:30,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:03:30,312 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 17:03:30,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:03:30,561 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 17:03:30,561 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 17:03:30,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 17:03:30.796 UTC [18795] WARNING: there is no transaction in progress
2026-08-08 17:03:30,802 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 17:03:30,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:03:31,039 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 17:03:31,039 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 17:03:31,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 17:03:31,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 17:03:31,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2j7r.c.gcp-upstream.internal','N') ;"
2026-08-08 17:03:31,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 17:03:32,047 [I] Table x_portal_user does not exist in database ranger
2026-08-08 17:03:32,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 17:03:32,295 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 17:03:32,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 17:03:32,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 17:03:32,754 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 17:03:32,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 17:03:33.666 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.681 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.697 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.710 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.725 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.743 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.801 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.808 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.822 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.831 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.841 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.854 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.863 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.869 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.877 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:33.883 UTC [18974] WARNING: there is no transaction in progress
2026-08-08 17:03:34,462 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 17:03:34,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 17:03:34,710 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 17:03:34,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 17:03:34,956 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 17:03:34,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 17:03:35,207 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 17:03:35,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 17:03:35,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 17:03:35,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2j7r.c.gcp-upstream.internal';"
2026-08-08 17:03:35,923 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 17:03:35,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 17:03:36,172 [I] DB_PATCHES have already been applied
I20260808 17:03:36.177469 31496 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-admin:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5D6OT5/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 17:03:36.177528 31496 mini_ranger.cc:193] Using host: 127.30.194.20
I20260808 17:03:36.181135 31496 mini_ranger.cc:227] Ranger admin URL: http://127.30.194.20:36885
Aug 08, 2026 5:03:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 5:03:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 5:03:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 5:03:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 5:03:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 5:03:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 5:03:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 17:03:37.378896 31496 mini_ranger.cc:160] Time spent starting Ranger: real 8.325s user 0.003s sys 0.006s
Aug 08, 2026 5:03:57 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 5:03:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 5:03:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 5:03:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 5:03:58 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 17:03:58.628453 31496 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/rangerkms_127.30.194.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/rangerkms_127.30.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.194.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.194.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.194.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/HTTP_127.30.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.194.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/HTTP_127.30.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 17:03:58 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208638, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 17:03:58.703065 31496 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 17:03:58.711970 31496 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 17:03:59.409787 31496 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 17:03:59.409859 31496 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-kms
I20260808 17:03:59.409885 31496 mini_ranger_kms.cc:227] Using postgres at 127.30.194.20:58067
2026-08-08 17:03:59,566 [I] DB FLAVOR :POSTGRES
2026-08-08 17:03:59,566 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 17:03:59,566 [I] Checking connection
2026-08-08 17:03:59,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 17:03:59,831 [I] connection success
2026-08-08 17:03:59,831 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 17:03:59,832 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 17:03:59,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 17:04:00,055 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 17:04:00,055 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 17:04:00,055 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 17:04:00,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.194.20:58067/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 17:04:00,296 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 17:04:00.811211 31496 mini_ranger_kms.cc:310] Created kms service
I20260808 17:04:01.026443 31496 mini_ranger_kms.cc:326] Created kudu user
I20260808 17:04:01.071870 31496 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 17:04:01 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208638, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:01.449455 31496 mini_ranger_kms.cc:384] Added ranger policy
I20260808 17:04:01.449680 31496 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-kms:/tmp/dist-test-task5D6OT5/build/release/bin/postgresql.jar:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5D6OT5/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 17:04:01.449704 31496 mini_ranger_kms.cc:240] Using host: 127.30.194.20
I20260808 17:04:01.454332 31496 mini_ranger_kms.cc:276] Ranger KMS PID: 19257
I20260808 17:04:01.454396 31496 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.194.20:47911
Aug 08, 2026 5:04:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 5:04:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 5:04:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 5:04:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 5:04:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 5:04:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 17:04:03 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208643, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.194.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:03.923323 31496 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.220s user 0.008s sys 0.130s
I20260808 17:04:03.923436 31496 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 17:04:03.923486 31496 mini_ranger_kms.cc:401] 127.30.194.20:47911/kms/v1/keys
Aug 08 17:04:04 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208643, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.194.20@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
Aug 08 17:04:04 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208643, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.194.20@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
Aug 08 17:04:06 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:04:06.783677 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43239
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:43239
--ranger_config_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:47911/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:06.865391 19336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:06.865702 19336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:06.865774 19336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:06.867364 19336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:06.867431 19336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:06.867452 19336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:06.867468 19336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:06.867484 19336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:06.869663 19336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.194.20:47911/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:43239
--ranger_config_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:06.869903 19336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:06.870128 19336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:04:06.875193 19336 server_base.cc:1034] running on GCE node
W20260808 17:04:06.875183 19350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:06.875312 19351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:06.875703 19353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:06.875970 19336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:06.876232 19336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:06.877364 19336 hybrid_clock.cc:648] HybridClock initialized: now 1786208646877344 us; error 38 us; skew 500 ppm
Aug 08 17:04:06 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208646, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:06.883285 19336 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:04:06.883682 19336 webserver.cc:460] Webserver started at http://127.30.194.62:44653/ using document root <none> and password file <none>
I20260808 17:04:06.883898 19336 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:06.883937 19336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:06.884053 19336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:04:06 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208646, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:06.942445 19336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "0e5cd1473be1465da299fb68631f64a5"
format_stamp: "Formatted at 2026-08-08 17:04:06 on dist-test-slave-2j7r"
server_key: "a2e66231e6d53263b8fec4516d36a6d3"
server_key_iv: "788e0f6d1fc9ea3fc72879690c7e0512"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:06.942826 19336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "0e5cd1473be1465da299fb68631f64a5"
format_stamp: "Formatted at 2026-08-08 17:04:06 on dist-test-slave-2j7r"
server_key: "a2e66231e6d53263b8fec4516d36a6d3"
server_key_iv: "788e0f6d1fc9ea3fc72879690c7e0512"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:06.944154 19336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:04:06.958173 19361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:06.958345 19336 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:04:06.958431 19336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "0e5cd1473be1465da299fb68631f64a5"
format_stamp: "Formatted at 2026-08-08 17:04:06 on dist-test-slave-2j7r"
server_key: "a2e66231e6d53263b8fec4516d36a6d3"
server_key_iv: "788e0f6d1fc9ea3fc72879690c7e0512"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:06.958503 19336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:06.968832 19336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:06.970283 19336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:06.970402 19336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:06.974282 19336 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:43239
I20260808 17:04:06.974367 19450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:43239 every 8 connection(s)
I20260808 17:04:06.974648 19336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:04:06.975317 19451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:06.977553 19451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: Bootstrap starting.
I20260808 17:04:06.978101 19451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:06.978358 19451 log.cc:824] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:06.978997 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 19336
I20260808 17:04:06.979064 19451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: No bootstrap required, opened a new log
I20260808 17:04:06.979110 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:04:06.979295 31496 external_mini_cluster.cc:1428] Setting key 88cc481bccff184992d4ee7b471c8cf9
I20260808 17:04:06.981274 19451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } }
I20260808 17:04:06.981413 19451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:06.981446 19451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e5cd1473be1465da299fb68631f64a5, State: Initialized, Role: FOLLOWER
I20260808 17:04:06.981568 19451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } }
I20260808 17:04:06.981637 19451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:04:06.981668 19451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:04:06.981717 19451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:06.982259 19451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } }
I20260808 17:04:06.982355 19451 leader_election.cc:302] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e5cd1473be1465da299fb68631f64a5; no voters:
I20260808 17:04:06.982512 19451 leader_election.cc:288] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:04:06.982542 19459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:04:06.982645 19459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 1 LEADER]: Becoming Leader. State: Replica: 0e5cd1473be1465da299fb68631f64a5, State: Running, Role: LEADER
I20260808 17:04:06.982728 19459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } }
I20260808 17:04:06.982758 19451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:04:06.983084 19459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e5cd1473be1465da299fb68631f64a5. Latest consensus state: current_term: 1 leader_uuid: "0e5cd1473be1465da299fb68631f64a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } } }
I20260808 17:04:06.983093 19461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e5cd1473be1465da299fb68631f64a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } } }
I20260808 17:04:06.983160 19459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:06.983184 19461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:06.984196 19451 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 17:04:06.986363 19465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:04:06.987399 19465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:04:06 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:06.993443 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:06.979772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.30.194.62:43239)
0808 17:04:06.979949 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:06.979958 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:06.979981 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:04:06.981257 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:06.981263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:06.981310 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:06.981430 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:06.982108 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:06.983274 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:06.984295 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:06.986857 (+ 2562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:06.987068 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:06.989825 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:06.989830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:06.989830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:06.989842 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:06.991649 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:06.992054 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:06.992057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:06.992058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:06.992183 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:06.992402 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:06.992402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:06.992403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:06.992746 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:06.992888 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:06.993027 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 17:04:06.993205 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260808 17:04:06.994522 19465 catalog_manager.cc:1347] Generated new cluster ID: eec8b8919721411fa6fbc9d9acbaec4d
I20260808 17:04:06.994580 19465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:04:07.011533 19465 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:04:07.012212 19465 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:04:07.016268 19465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: Generated new TSK 0
I20260808 17:04:07.016431 19465 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:04:07.682183 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:47911/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43239
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:07.769848 19529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:07.770032 19529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:07.770053 19529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:07.771147 19529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:07.771186 19529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:07.771209 19529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:04:07.772616 19529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.194.20:47911/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:07.772801 19529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:07.773000 19529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:07.778666 19539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:07.778677 19536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:07.778926 19543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:07.779044 19529 server_base.cc:1034] running on GCE node
I20260808 17:04:07.779260 19529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:07.779520 19529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:07.780699 19529 hybrid_clock.cc:648] HybridClock initialized: now 1786208647780680 us; error 33 us; skew 500 ppm
Aug 08 17:04:07 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208647, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:07.783742 19529 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:04:07.784185 19529 webserver.cc:460] Webserver started at http://127.30.194.1:37317/ using document root <none> and password file <none>
I20260808 17:04:07.784394 19529 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:07.784442 19529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:07.784587 19529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:04:07 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208647, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:07.797839 19529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "8de09fe68915482a8990468a04386b34"
format_stamp: "Formatted at 2026-08-08 17:04:07 on dist-test-slave-2j7r"
server_key: "8698f1a75f78c1376655d35806e6a31a"
server_key_iv: "641fc0dfebfebbbcc06c354ff97f3f34"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:07.798300 19529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "8de09fe68915482a8990468a04386b34"
format_stamp: "Formatted at 2026-08-08 17:04:07 on dist-test-slave-2j7r"
server_key: "8698f1a75f78c1376655d35806e6a31a"
server_key_iv: "641fc0dfebfebbbcc06c354ff97f3f34"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:07.800141 19529 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 17:04:07.812944 19553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:07.813195 19529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:04:07.813287 19529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "8de09fe68915482a8990468a04386b34"
format_stamp: "Formatted at 2026-08-08 17:04:07 on dist-test-slave-2j7r"
server_key: "8698f1a75f78c1376655d35806e6a31a"
server_key_iv: "641fc0dfebfebbbcc06c354ff97f3f34"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:07.813357 19529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:07.842327 19529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:07.844168 19529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:07.844310 19529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:07.844508 19529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:07.844866 19529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:07.844902 19529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:07.844933 19529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:07.844957 19529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:07.851986 19529 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:36305
I20260808 17:04:07.852371 19529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:04:07.852677 19736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:36305 every 8 connection(s)
I20260808 17:04:07.857632 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 19529
I20260808 17:04:07.857743 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:04:07.857954 31496 external_mini_cluster.cc:1428] Setting key acb2db8d7552eb1d4c7ff9722ccc8930
Aug 08 17:04:07 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208647, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:07.862912 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:07.853123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:41213 (local address 127.30.194.62:43239)
0808 17:04:07.854129 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:07.854132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:07.854140 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:04:07.854165 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:07.854165 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:07.854176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:07.854239 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:07.854785 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:07.855591 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:07.857076 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:07.857358 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:07.857421 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:07.859774 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:07.859776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:07.859777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:07.859785 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:07.861448 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:07.861839 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:07.861840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:07.861841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:07.861905 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:07.862173 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:07.862174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:07.862174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:07.862321 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:07.862401 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:07.862711 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 17:04:07.862816 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":972,"thread_start_us":80,"threads_started":1}
I20260808 17:04:07.863036 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:07.853064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43239 (local address 127.30.194.1:41213)
0808 17:04:07.853300 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:07.853343 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:07.853934 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:07.854336 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:07.854340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:07.854510 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:07.854690 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:07.854694 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:07.855719 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:07.855720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:07.856971 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:07.856974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:07.857455 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:07.857457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:07.857578 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:07.858071 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:07.858080 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:07.859677 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:07.861518 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:07.861531 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:07.861533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:07.861772 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:07.861955 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:07.861956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:07.861956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:07.862107 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:07.862397 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:07.862404 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:07.862556 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 17:04:07.862664 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 17:04:07.862810 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
I20260808 17:04:07.863565 19738 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43239
I20260808 17:04:07.863677 19738 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:07.863888 19738 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
I20260808 17:04:07.864511 19386 ts_manager.cc:194] Registered new tserver with Master: 8de09fe68915482a8990468a04386b34 (127.30.194.1:36305)
I20260808 17:04:07.865408 19386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:41213
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:04:07.885957 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:47911/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43239
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:07.994374 19747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:07.994632 19747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:07.994669 19747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:07.996306 19747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:07.996359 19747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:07.996397 19747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:04:07.998456 19747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.194.20:47911/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:07.998679 19747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:07.998924 19747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:08.004977 19773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:08.005051 19768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:08.005347 19747 server_base.cc:1034] running on GCE node
W20260808 17:04:08.005594 19769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:08.005821 19747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:08.006106 19747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:08.007290 19747 hybrid_clock.cc:648] HybridClock initialized: now 1786208648007257 us; error 42 us; skew 500 ppm
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:08.010532 19747 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:04:08.011132 19747 webserver.cc:460] Webserver started at http://127.30.194.2:45261/ using document root <none> and password file <none>
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:08.011857 19747 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:08.011929 19747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:08.012060 19747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:08.037266 19747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "d4dbc0280d55411d9e1b444fc007614d"
format_stamp: "Formatted at 2026-08-08 17:04:08 on dist-test-slave-2j7r"
server_key: "cf2327ee39e332c946d1854f4e22dcd6"
server_key_iv: "a2aa0cbca2b1f2a1eca918a4637d8576"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:08.037710 19747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "d4dbc0280d55411d9e1b444fc007614d"
format_stamp: "Formatted at 2026-08-08 17:04:08 on dist-test-slave-2j7r"
server_key: "cf2327ee39e332c946d1854f4e22dcd6"
server_key_iv: "a2aa0cbca2b1f2a1eca918a4637d8576"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:08.039541 19747 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 17:04:08.052629 19786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.052825 19747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:08.052906 19747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "d4dbc0280d55411d9e1b444fc007614d"
format_stamp: "Formatted at 2026-08-08 17:04:08 on dist-test-slave-2j7r"
server_key: "cf2327ee39e332c946d1854f4e22dcd6"
server_key_iv: "a2aa0cbca2b1f2a1eca918a4637d8576"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:08.052973 19747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:08.080163 19747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:08.082504 19747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:08.082720 19747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:08.083033 19747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:08.083420 19747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:08.083468 19747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.083498 19747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:08.083520 19747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.094944 19747 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:40317
I20260808 17:04:08.095369 19747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:04:08.095969 19998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:40317 every 8 connection(s)
I20260808 17:04:08.101330 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 19747
I20260808 17:04:08.101432 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:04:08.101621 31496 external_mini_cluster.cc:1428] Setting key e5090dc413c918e36cfbaf656408f6fc
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:08.109995 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:08.097131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:49641 (local address 127.30.194.62:43239)
0808 17:04:08.097180 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:08.097182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:08.097514 (+ 332us) server_negotiation.cc:407] Connection header received
0808 17:04:08.098224 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:08.098225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:08.098243 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:08.098273 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:08.099076 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.099862 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.101328 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.101604 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.101650 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:08.106751 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:08.106753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:08.106753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:08.106760 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:08.108402 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:08.108866 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:08.108867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:08.108868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:08.108926 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:08.109225 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:08.109226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:08.109227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:08.109375 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:08.109428 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:08.109794 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 17:04:08.109883 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:04:08.110795 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:08.096834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43239 (local address 127.30.194.2:49641)
0808 17:04:08.097388 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:08.097431 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:08.098069 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:08.098467 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:08.098471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:08.098701 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:08.098968 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:08.098974 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.099975 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.099976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:08.101216 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:08.101219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.101756 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.101757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:08.104388 (+ 2631us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:08.104943 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:08.104954 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:08.106637 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:08.108497 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:08.108504 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:08.108506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:08.108776 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:08.108989 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:08.108990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:08.108990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:08.109143 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:08.109463 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:08.109469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:08.109627 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 17:04:08.110415 (+ 788us) client_negotiation.cc:240] Negotiation successful
0808 17:04:08.110569 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":82,"threads_started":1}
I20260808 17:04:08.111400 19999 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43239
I20260808 17:04:08.111490 19999 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:08.111670 19999 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
I20260808 17:04:08.112079 19386 ts_manager.cc:194] Registered new tserver with Master: d4dbc0280d55411d9e1b444fc007614d (127.30.194.2:40317)
I20260808 17:04:08.112553 19386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:49641
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:04:08.142053 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:47911/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43239
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:08.255092 20009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:08.255414 20009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:08.255501 20009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:08.257172 20009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:08.257275 20009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:08.257360 20009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:04:08.259544 20009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.194.20:47911/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:08.259814 20009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:08.260128 20009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:08.267637 20022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:08.267834 20024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:08.268841 20021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:08.272161 20009 server_base.cc:1034] running on GCE node
I20260808 17:04:08.272425 20009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:08.272722 20009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:08.273885 20009 hybrid_clock.cc:648] HybridClock initialized: now 1786208648273786 us; error 109 us; skew 500 ppm
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:08.277478 20009 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:04:08.277993 20009 webserver.cc:460] Webserver started at http://127.30.194.3:44429/ using document root <none> and password file <none>
I20260808 17:04:08.278352 20009 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:08.278429 20009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:08.278573 20009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:08.295795 20009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7"
format_stamp: "Formatted at 2026-08-08 17:04:08 on dist-test-slave-2j7r"
server_key: "86cbf76960711ad3d27bb3e3688770ed"
server_key_iv: "7e039639d3dc84c7753a44806c25518d"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:08.296307 20009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7"
format_stamp: "Formatted at 2026-08-08 17:04:08 on dist-test-slave-2j7r"
server_key: "86cbf76960711ad3d27bb3e3688770ed"
server_key_iv: "7e039639d3dc84c7753a44806c25518d"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:08.301333 20009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260808 17:04:08.313122 20037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.313573 20009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:08.313741 20009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7"
format_stamp: "Formatted at 2026-08-08 17:04:08 on dist-test-slave-2j7r"
server_key: "86cbf76960711ad3d27bb3e3688770ed"
server_key_iv: "7e039639d3dc84c7753a44806c25518d"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:08.313895 20009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:08.353962 20009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:08.356227 20009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:08.356475 20009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:08.356774 20009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:08.357215 20009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:08.357302 20009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.357375 20009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:08.357430 20009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.363709 20009 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:41689
I20260808 17:04:08.364171 20009 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:04:08.369836 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 20009
I20260808 17:04:08.369973 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:04:08.370158 31496 external_mini_cluster.cc:1428] Setting key ace1dd434a5b30f9f85199c942ad5ac7
I20260808 17:04:08.378536 20228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:41689 every 8 connection(s)
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:08.393651 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:08.379505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:43079 (local address 127.30.194.62:43239)
0808 17:04:08.379544 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:08.379546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:08.379554 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:04:08.380205 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:08.380206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:08.380217 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:08.380244 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:08.383194 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.383960 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.386756 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.387067 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.387106 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:08.390363 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:08.390365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:08.390365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:08.390373 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:08.391886 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:08.392613 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:08.392614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:08.392614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:08.392681 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:08.392954 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:08.392954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:08.392955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:08.393079 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:08.393148 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:08.393474 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 17:04:08.393556 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:04:08.394353 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:08.379198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43239 (local address 127.30.194.3:43079)
0808 17:04:08.379392 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:08.379438 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:08.380068 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:08.380429 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:08.380434 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:08.382800 (+ 2366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:08.383086 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:08.383092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.384311 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.384312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:08.386656 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:08.386660 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.387747 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.387748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:08.387908 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:08.388438 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:08.388449 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:08.390259 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:08.392252 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:08.392258 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:08.392261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:08.392530 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:08.392748 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:08.392748 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:08.392749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:08.392886 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:08.393178 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:08.393184 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:08.393333 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 17:04:08.393978 (+ 645us) client_negotiation.cc:240] Negotiation successful
0808 17:04:08.394130 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
I20260808 17:04:08.395089 20229 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43239
I20260808 17:04:08.395175 20229 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:08.395360 20229 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
I20260808 17:04:08.395741 19386 ts_manager.cc:194] Registered new tserver with Master: 6fecc9b5876e4ebcb129ef78d46c6cc7 (127.30.194.3:41689)
I20260808 17:04:08.396194 19386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:43079
I20260808 17:04:08.402912 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:04:08.403556 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 19529
I20260808 17:04:08.455235 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 19747
I20260808 17:04:08.467793 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 20009
I20260808 17:04:08.475850 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 19336
I20260808 17:04:08.490887 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43239
--webserver_interface=127.30.194.62
--webserver_port=44653
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:43239
--ranger_config_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:47911/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.194.20:47911/kms with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:08.572036 20235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:08.572222 20235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:08.572244 20235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:08.573370 20235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:08.573410 20235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:08.573421 20235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:08.573432 20235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:08.573443 20235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:08.575023 20235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.194.20:47911/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:43239
--ranger_config_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=44653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:08.575233 20235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:08.575433 20235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:08.581163 20248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:08.581185 20246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:08.581300 20235 server_base.cc:1034] running on GCE node
W20260808 17:04:08.581449 20244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:08.581686 20235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:08.581970 20235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:08.583112 20235 hybrid_clock.cc:648] HybridClock initialized: now 1786208648583098 us; error 31 us; skew 500 ppm
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:08.585723 20235 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:04:08.586252 20235 webserver.cc:460] Webserver started at http://127.30.194.62:44653/ using document root <none> and password file <none>
I20260808 17:04:08.586488 20235 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:08.586555 20235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:08.630297 20235 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:08.630960 20262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.631122 20235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:08.631215 20235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "0e5cd1473be1465da299fb68631f64a5"
format_stamp: "Formatted at 2026-08-08 17:04:06 on dist-test-slave-2j7r"
server_key: "a2e66231e6d53263b8fec4516d36a6d3"
server_key_iv: "788e0f6d1fc9ea3fc72879690c7e0512"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:08.631561 20235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:08.642349 20235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:08.644205 20235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:08.644346 20235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:08.649220 20235 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:43239
I20260808 17:04:08.649271 20340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:43239 every 8 connection(s)
I20260808 17:04:08.649684 20235 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:04:08.650894 20341 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:04:08.651508 20341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: Bootstrap starting.
I20260808 17:04:08.656792 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 20235
I20260808 17:04:08.657176 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:47911/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:36305
--local_ip_for_outbound_sockets=127.30.194.1
--tserver_master_addrs=127.30.194.62:43239
--webserver_port=37317
--webserver_interface=127.30.194.1
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.194.20:47911/kms with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:08.659557 20341 log.cc:824] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:08.661490 20341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:04:08.661760 20341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: Bootstrap complete.
I20260808 17:04:08.665107 20341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } }
I20260808 17:04:08.665437 20341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e5cd1473be1465da299fb68631f64a5, State: Initialized, Role: FOLLOWER
I20260808 17:04:08.665608 20341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } }
I20260808 17:04:08.665680 20341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:04:08.665720 20341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:04:08.665758 20341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:04:08.667093 20341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } }
I20260808 17:04:08.667207 20341 leader_election.cc:302] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e5cd1473be1465da299fb68631f64a5; no voters:
I20260808 17:04:08.667376 20341 leader_election.cc:288] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 17:04:08.667490 20348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:04:08.667629 20341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:04:08.667752 20348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [term 2 LEADER]: Becoming Leader. State: Replica: 0e5cd1473be1465da299fb68631f64a5, State: Running, Role: LEADER
I20260808 17:04:08.667896 20348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } }
I20260808 17:04:08.668207 20348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0e5cd1473be1465da299fb68631f64a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } } }
I20260808 17:04:08.668340 20348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:08.668509 20348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e5cd1473be1465da299fb68631f64a5. Latest consensus state: current_term: 2 leader_uuid: "0e5cd1473be1465da299fb68631f64a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e5cd1473be1465da299fb68631f64a5" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43239 } } }
I20260808 17:04:08.668606 20348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5 [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:08.668762 20364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:04:08.669142 20341 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 17:04:08.669404 20364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:04:08.673408 20364 catalog_manager.cc:1259] Loaded cluster ID: eec8b8919721411fa6fbc9d9acbaec4d
I20260808 17:04:08.673578 20364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:04:08.674817 20364 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:04:08.675009 20364 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: Loaded TSK: 0
I20260808 17:04:08.675231 20364 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:04:08.754067 20343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:08.754333 20343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:08.754374 20343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:08.755493 20343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:08.755533 20343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:08.755566 20343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:04:08.757050 20343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.194.20:47911/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:36305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=37317
--tserver_master_addrs=127.30.194.62:43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:08.757297 20343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:08.757501 20343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:08.762930 20413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:08.763000 20418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:08.763262 20343 server_base.cc:1034] running on GCE node
W20260808 17:04:08.763442 20414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:08.763682 20343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:08.763919 20343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:08.767143 20343 hybrid_clock.cc:648] HybridClock initialized: now 1786208648767125 us; error 25 us; skew 500 ppm
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:08.771865 20343 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:04:08.772365 20343 webserver.cc:460] Webserver started at http://127.30.194.1:37317/ using document root <none> and password file <none>
I20260808 17:04:08.772598 20343 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:08.772663 20343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:08.793180 20343 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 17:04:08.797020 20438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.797207 20343 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260808 17:04:08.797294 20343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "8de09fe68915482a8990468a04386b34"
format_stamp: "Formatted at 2026-08-08 17:04:07 on dist-test-slave-2j7r"
server_key: "8698f1a75f78c1376655d35806e6a31a"
server_key_iv: "641fc0dfebfebbbcc06c354ff97f3f34"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:08.797747 20343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:08.815013 20343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:08.817283 20343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:08.817459 20343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:08.817683 20343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:08.818063 20343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:08.818117 20343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.818147 20343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:08.818171 20343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.827590 20343 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:36305
I20260808 17:04:08.828025 20343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:04:08.829057 20629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:36305 every 8 connection(s)
I20260808 17:04:08.835435 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 20343
I20260808 17:04:08.835726 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:47911/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:40317
--local_ip_for_outbound_sockets=127.30.194.2
--tserver_master_addrs=127.30.194.62:43239
--webserver_port=45261
--webserver_interface=127.30.194.2
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.194.20:47911/kms with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:08.847780 20633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:08.829174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43239 (local address 127.30.194.1:44499)
0808 17:04:08.833467 (+ 4293us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:08.833515 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:08.834088 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:08.834744 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:08.834749 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:08.835074 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:08.835402 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:08.835410 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.838295 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.838296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:08.840240 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:08.840244 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.841250 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.841251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:08.841378 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:08.841989 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:08.842000 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:08.843505 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:08.845790 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:08.845807 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:08.845809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:08.846121 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:08.846424 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:08.846425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:08.846425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:08.846567 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:08.847101 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:08.847109 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:08.847267 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 17:04:08.847392 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 17:04:08.847550 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4247,"thread_start_us":96,"threads_started":1}
I20260808 17:04:08.848313 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:08.829599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:44499 (local address 127.30.194.62:43239)
0808 17:04:08.834275 (+ 4676us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:08.834285 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:08.834324 (+ 39us) server_negotiation.cc:407] Connection header received
0808 17:04:08.834461 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:08.834467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:08.834518 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:08.834641 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:08.835596 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.838154 (+ 2558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.840351 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:08.841001 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:08.841130 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:08.843632 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:08.843637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:08.843638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:08.843651 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:08.845695 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:08.846216 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:08.846220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:08.846220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:08.846341 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:08.846654 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:08.846655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:08.846655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:08.846983 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:08.847068 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:08.847484 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 17:04:08.847614 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4621,"thread_start_us":96,"threads_started":1}
I20260808 17:04:08.849148 20631 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43239
I20260808 17:04:08.849237 20631 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:08.849416 20631 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
W20260808 17:04:08.850346 20631 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:43239 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 17:04:08.850435 20631 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:08.850513 20631 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
I20260808 17:04:08.850921 20631 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:08.851001 20631 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
W20260808 17:04:08.851357 20631 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:04:08.940588 20636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:08.940836 20636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:08.940868 20636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:08.942415 20636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:08.942473 20636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:08.942509 20636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:04:08.944140 20636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.194.20:47911/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:40317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=45261
--tserver_master_addrs=127.30.194.62:43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:08.944387 20636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:08.944633 20636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:08.950526 20642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:08.950615 20646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:08.950831 20643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:08.951589 20636 server_base.cc:1034] running on GCE node
I20260808 17:04:08.951771 20636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:08.952006 20636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:08.953122 20636 hybrid_clock.cc:648] HybridClock initialized: now 1786208648953044 us; error 92 us; skew 500 ppm
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:08.955463 20636 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:04:08.955905 20636 webserver.cc:460] Webserver started at http://127.30.194.2:45261/ using document root <none> and password file <none>
I20260808 17:04:08.956130 20636 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:08.956188 20636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 17:04:08 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:08.975442 20636 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 17:04:08.977205 20658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:08.977350 20636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 17:04:08.977434 20636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "d4dbc0280d55411d9e1b444fc007614d"
format_stamp: "Formatted at 2026-08-08 17:04:08 on dist-test-slave-2j7r"
server_key: "cf2327ee39e332c946d1854f4e22dcd6"
server_key_iv: "a2aa0cbca2b1f2a1eca918a4637d8576"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:08.978566 20636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:09.008491 20636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:09.010772 20636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:09.010905 20636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:09.011106 20636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:09.011466 20636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:09.011514 20636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:09.011543 20636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:09.011569 20636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:09.018810 20636 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:40317
I20260808 17:04:09.019275 20636 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:04:09.019897 20840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:40317 every 8 connection(s)
I20260808 17:04:09.024936 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 20636
Aug 08 17:04:09 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:09.025295 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.194.20:47911/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:41689
--local_ip_for_outbound_sockets=127.30.194.3
--tserver_master_addrs=127.30.194.62:43239
--webserver_port=44429
--webserver_interface=127.30.194.3
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.194.20:47911/kms with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:09.030260 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:09.020070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43239 (local address 127.30.194.2:58659)
0808 17:04:09.020205 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:09.020244 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:09.020720 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:09.020960 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:09.020965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:09.021206 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:09.021469 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:09.021476 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:09.022471 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:09.022472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:09.023235 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:09.023238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:09.023727 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:09.023727 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:09.023827 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:09.024263 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:09.024271 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:09.025798 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:09.028558 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:09.028565 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:09.028568 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:09.028870 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:09.029109 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:09.029110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:09.029111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:09.029265 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:09.029593 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:09.029600 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:09.029756 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 17:04:09.029866 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 17:04:09.030016 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":76,"threads_started":1}
I20260808 17:04:09.030690 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:09.020347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:58659 (local address 127.30.194.62:43239)
0808 17:04:09.020390 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:09.020392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:09.020401 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:04:09.020825 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:09.020827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:09.020840 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:09.020890 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:09.021588 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:09.022353 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:09.023324 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:09.023632 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:09.023713 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:09.025892 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:09.025894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:09.025894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:09.025901 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:09.028463 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:09.028951 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:09.028952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:09.028953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:09.029016 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:09.029357 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:09.029358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:09.029358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:09.029498 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:09.029562 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:09.030504 (+ 942us) server_negotiation.cc:299] Negotiation successful
0808 17:04:09.030599 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:04:09.031036 20843 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43239
I20260808 17:04:09.031142 20843 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:09.031344 20843 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
W20260808 17:04:09.032094 20843 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:43239 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 17:04:09.032166 20843 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:09.032238 20843 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
I20260808 17:04:09.032702 20843 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:09.032780 20843 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
W20260808 17:04:09.033766 20843 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:04:09.126066 20849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:09.126308 20849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:09.126343 20849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:09.127723 20849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:09.127769 20849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:09.127812 20849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:04:09.129287 20849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.194.20:47911/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:41689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=44429
--tserver_master_addrs=127.30.194.62:43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:09.129513 20849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:09.129711 20849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:09.134423 20858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:09.134708 20857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:09.134943 20860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:09.135035 20849 server_base.cc:1034] running on GCE node
I20260808 17:04:09.135267 20849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:09.135505 20849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:09.136649 20849 hybrid_clock.cc:648] HybridClock initialized: now 1786208649136638 us; error 24 us; skew 500 ppm
Aug 08 17:04:09 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208649, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:09.139073 20849 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:04:09.139500 20849 webserver.cc:460] Webserver started at http://127.30.194.3:44429/ using document root <none> and password file <none>
I20260808 17:04:09.139748 20849 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:09.139813 20849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 17:04:09 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208649, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:09.157446 20849 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 17:04:09.165469 20877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:09.165648 20849 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.000s
I20260808 17:04:09.165736 20849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7"
format_stamp: "Formatted at 2026-08-08 17:04:08 on dist-test-slave-2j7r"
server_key: "86cbf76960711ad3d27bb3e3688770ed"
server_key_iv: "7e039639d3dc84c7753a44806c25518d"
server_key_version: "kuduclusterkey@0"
I20260808 17:04:09.166079 20849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:09.178853 20849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:09.180954 20849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:09.181126 20849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:09.181353 20849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:09.181641 20849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:09.181675 20849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:09.181702 20849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:09.181727 20849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:09.187930 20849 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:41689
I20260808 17:04:09.187991 21048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:41689 every 8 connection(s)
I20260808 17:04:09.188395 20849 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:04:09.192452 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 20849
Aug 08 17:04:09 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208649, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:09.198724 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:09.189779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:59489 (local address 127.30.194.62:43239)
0808 17:04:09.189824 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:09.189826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:09.190086 (+ 260us) server_negotiation.cc:407] Connection header received
0808 17:04:09.190681 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:09.190683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:09.190697 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:09.190750 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:09.191421 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:09.192183 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:09.193731 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:09.194036 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:09.194112 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:09.196134 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:09.196135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:09.196136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:09.196143 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:09.197529 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:09.197872 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:09.197873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:09.197874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:09.197930 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:09.198138 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:09.198139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:09.198140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:09.198251 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:09.198306 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:09.198551 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 17:04:09.198633 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:04:09.198982 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:09.189785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43239 (local address 127.30.194.3:59489)
0808 17:04:09.189948 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:09.189993 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:09.190548 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:09.190837 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:09.190841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:09.191049 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:09.191319 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:09.191326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:09.192285 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:09.192286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:09.193640 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:09.193642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:09.194155 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:09.194156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:09.194269 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:09.194681 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:09.194690 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:09.196055 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:09.197589 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:09.197596 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:09.197598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:09.197808 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:09.197981 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:09.197982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:09.197982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:09.198084 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:09.198304 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:09.198310 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:09.198435 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 17:04:09.198521 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 17:04:09.198651 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260808 17:04:09.199725 21049 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43239
I20260808 17:04:09.199820 21049 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:09.199982 21049 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
W20260808 17:04:09.200706 21049 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:43239 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 17:04:09.200771 21049 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:09.200837 21049 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
I20260808 17:04:09.201188 21049 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:09.201274 21049 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
W20260808 17:04:09.201584 21049 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:04:09.201813 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:09.192867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51736 (local address 127.30.194.62:43239)
0808 17:04:09.193757 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:09.193759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:09.193767 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:04:09.193787 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:09.193788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:09.193805 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:09.193864 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:09.194905 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:09.195480 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:09.197048 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:09.197374 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:09.197446 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:09.199097 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:09.199099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:09.199099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:09.199106 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:09.200540 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:09.200911 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:09.200912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:09.200912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:09.200972 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:09.201195 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:09.201196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:09.201196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:09.201400 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:09.201551 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:09.201602 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 17:04:09.201728 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":53,"threads_started":1}
Aug 08 17:04:09 dist-test-slave-2j7r krb5kdc[18614](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208649, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:09.851529 20631 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:09.851689 20631 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
I20260808 17:04:09.852097 20287 ts_manager.cc:194] Registered new tserver with Master: 8de09fe68915482a8990468a04386b34 (127.30.194.1:36305)
I20260808 17:04:09.853775 20287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:44499
I20260808 17:04:10.033926 20843 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:10.034070 20843 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
I20260808 17:04:10.034329 20287 ts_manager.cc:194] Registered new tserver with Master: d4dbc0280d55411d9e1b444fc007614d (127.30.194.2:40317)
I20260808 17:04:10.034886 20287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:58659
Aug 08 17:04:10 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208649, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:10.201782 21049 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:10.201931 21049 heartbeater.cc:511] Master 127.30.194.62:43239 requested a full tablet report, sending...
I20260808 17:04:10.202286 20287 ts_manager.cc:194] Registered new tserver with Master: 6fecc9b5876e4ebcb129ef78d46c6cc7 (127.30.194.3:41689)
I20260808 17:04:10.202798 20287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:59489
I20260808 17:04:10.205171 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:04:10.214076 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.206199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.30.194.62:43239)
0808 17:04:10.206470 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:10.206472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:10.206624 (+ 152us) server_negotiation.cc:407] Connection header received
0808 17:04:10.206645 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:10.206646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:10.206656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:10.206709 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:10.207399 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.208171 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.209210 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.209531 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.209584 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.211019 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:10.211021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:10.211021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:10.211029 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:10.212648 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:10.213009 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:10.213010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:10.213010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:10.213064 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:10.213296 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:10.213297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:10.213297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:10.213419 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:10.213503 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:10.213892 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 17:04:10.213973 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":86,"threads_started":1}
I20260808 17:04:10.216393 20287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:04:10.218588 20287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:04:10.237403 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.221792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:40317 (local address 127.0.0.1:38454)
0808 17:04:10.225448 (+ 3656us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:10.225455 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:10.225507 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:10.227978 (+ 2471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:10.227980 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:10.227982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:10.228033 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:10.228034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.233184 (+ 5150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.233185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:10.235624 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:10.235627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.237051 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.237052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:10.237185 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.237202 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 17:04:10.237274 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:04:10.237321 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3643,"thread_start_us":66,"threads_started":1}
I20260808 17:04:10.237799 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.221902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38454 (local address 127.30.194.2:40317)
0808 17:04:10.225208 (+ 3306us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:10.225213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:10.225580 (+ 367us) server_negotiation.cc:407] Connection header received
0808 17:04:10.225619 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:10.225621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:10.225637 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:10.225697 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:10.228105 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.229137 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.235708 (+ 6571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.236451 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.236515 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.236567 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:10.237683 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0808 17:04:10.237724 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3268,"thread_start_us":52,"threads_started":1}
I20260808 17:04:10.239485 21099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.222063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59216 (local address 127.30.194.3:41689)
0808 17:04:10.222695 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:10.222699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:10.225910 (+ 3211us) server_negotiation.cc:407] Connection header received
0808 17:04:10.225964 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:10.225967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:10.225981 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:10.226045 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:10.226821 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.227888 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.232044 (+ 4156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.234977 (+ 2933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.235043 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.235104 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:10.239366 (+ 4262us) server_negotiation.cc:299] Negotiation successful
0808 17:04:10.239408 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":65,"threads_started":1}
I20260808 17:04:10.239923 21100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.222187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41492 (local address 127.30.194.1:36305)
0808 17:04:10.222606 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:10.222611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:10.225403 (+ 2792us) server_negotiation.cc:407] Connection header received
0808 17:04:10.226204 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:10.226207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:10.226222 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:10.226282 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:10.229570 (+ 3288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.230610 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.238011 (+ 7401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.238754 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.238800 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.238848 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:10.239801 (+ 953us) server_negotiation.cc:299] Negotiation successful
0808 17:04:10.239846 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":55,"threads_started":1}
I20260808 17:04:10.239301 21098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.221944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:41689 (local address 127.0.0.1:59216)
0808 17:04:10.225806 (+ 3862us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:10.225814 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:10.225857 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:10.226681 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:10.226682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:10.226683 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:10.226746 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:10.226747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.230683 (+ 3936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.230684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:10.231950 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:10.231953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.239088 (+ 7135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.239089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:10.239141 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.239144 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:04:10.239185 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 17:04:10.239220 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3848,"thread_start_us":83,"threads_started":1}
I20260808 17:04:10.240242 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.221579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:36305 (local address 127.0.0.1:41492)
0808 17:04:10.225295 (+ 3716us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:10.225337 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:10.226114 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:10.226395 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:10.226399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:10.226401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:10.229484 (+ 3083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:10.229486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.232835 (+ 3349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.232836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:10.236985 (+ 4149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:10.236987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.238906 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.238907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:10.238971 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.238974 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:04:10.240140 (+ 1166us) client_negotiation.cc:240] Negotiation successful
0808 17:04:10.240179 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3685,"thread_start_us":85,"threads_started":1}
I20260808 17:04:10.241025 20707 tablet_service.cc:1467] Processing CreateTablet for tablet 115f570887bc49a19a28477595c9b56a (DEFAULT_TABLE table=test-table [id=674d56eabc29464289c10254c2a3262e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:10.241454 20707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 115f570887bc49a19a28477595c9b56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:10.241367 20949 tablet_service.cc:1467] Processing CreateTablet for tablet 115f570887bc49a19a28477595c9b56a (DEFAULT_TABLE table=test-table [id=674d56eabc29464289c10254c2a3262e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:10.241741 20949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 115f570887bc49a19a28477595c9b56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:10.241675 20511 tablet_service.cc:1467] Processing CreateTablet for tablet 115f570887bc49a19a28477595c9b56a (DEFAULT_TABLE table=test-table [id=674d56eabc29464289c10254c2a3262e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:10.242024 20511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 115f570887bc49a19a28477595c9b56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:10.246129 21104 tablet_bootstrap.cc:492] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34: Bootstrap starting.
I20260808 17:04:10.246802 21104 tablet_bootstrap.cc:654] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:10.247161 21104 log.cc:824] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:10.247928 21104 tablet_bootstrap.cc:492] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34: No bootstrap required, opened a new log
I20260808 17:04:10.247989 21104 ts_tablet_manager.cc:1397] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:04:10.250492 21104 raft_consensus.cc:348] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 } } peers { permanent_uuid: "d4dbc0280d55411d9e1b444fc007614d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40317 } } peers { permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 } }
I20260808 17:04:10.250659 21104 raft_consensus.cc:374] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:10.250689 21104 raft_consensus.cc:729] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de09fe68915482a8990468a04386b34, State: Initialized, Role: FOLLOWER
I20260808 17:04:10.250813 21104 consensus_queue.cc:260] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 } } peers { permanent_uuid: "d4dbc0280d55411d9e1b444fc007614d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40317 } } peers { permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 } }
I20260808 17:04:10.251055 21104 ts_tablet_manager.cc:1428] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:04:10.251166 20631 heartbeater.cc:503] Master 127.30.194.62:43239 was elected leader, sending a full tablet report...
I20260808 17:04:10.253227 21101 tablet_bootstrap.cc:492] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: Bootstrap starting.
I20260808 17:04:10.253252 21103 tablet_bootstrap.cc:492] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: Bootstrap starting.
I20260808 17:04:10.253906 21101 tablet_bootstrap.cc:654] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:10.253978 21103 tablet_bootstrap.cc:654] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:10.254273 21101 log.cc:824] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:10.254355 21103 log.cc:824] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:10.255018 21101 tablet_bootstrap.cc:492] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: No bootstrap required, opened a new log
I20260808 17:04:10.255079 21101 ts_tablet_manager.cc:1397] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:04:10.255111 21103 tablet_bootstrap.cc:492] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: No bootstrap required, opened a new log
I20260808 17:04:10.255170 21103 ts_tablet_manager.cc:1397] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:04:10.257504 21101 raft_consensus.cc:348] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 } } peers { permanent_uuid: "d4dbc0280d55411d9e1b444fc007614d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40317 } } peers { permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 } }
I20260808 17:04:10.257654 21101 raft_consensus.cc:374] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:10.257684 21101 raft_consensus.cc:729] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbc0280d55411d9e1b444fc007614d, State: Initialized, Role: FOLLOWER
I20260808 17:04:10.257578 21103 raft_consensus.cc:348] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 } } peers { permanent_uuid: "d4dbc0280d55411d9e1b444fc007614d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40317 } } peers { permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 } }
I20260808 17:04:10.257730 21103 raft_consensus.cc:374] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:10.257766 21103 raft_consensus.cc:729] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fecc9b5876e4ebcb129ef78d46c6cc7, State: Initialized, Role: FOLLOWER
I20260808 17:04:10.257781 21101 consensus_queue.cc:260] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 } } peers { permanent_uuid: "d4dbc0280d55411d9e1b444fc007614d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40317 } } peers { permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 } }
I20260808 17:04:10.257918 21103 consensus_queue.cc:260] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 } } peers { permanent_uuid: "d4dbc0280d55411d9e1b444fc007614d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40317 } } peers { permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 } }
I20260808 17:04:10.258023 21101 ts_tablet_manager.cc:1428] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:04:10.258181 21103 ts_tablet_manager.cc:1428] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:04:10.258316 21049 heartbeater.cc:503] Master 127.30.194.62:43239 was elected leader, sending a full tablet report...
I20260808 17:04:10.258358 20843 heartbeater.cc:503] Master 127.30.194.62:43239 was elected leader, sending a full tablet report...
W20260808 17:04:10.270732 20844 tablet.cc:2369] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:04:10.330502 20632 tablet.cc:2369] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:04:10.339641 21115 raft_consensus.cc:482] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:10.339871 21115 raft_consensus.cc:504] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 } } peers { permanent_uuid: "d4dbc0280d55411d9e1b444fc007614d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40317 } } peers { permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 } }
I20260808 17:04:10.340324 21115 leader_election.cc:288] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8de09fe68915482a8990468a04386b34 (127.30.194.1:36305), 6fecc9b5876e4ebcb129ef78d46c6cc7 (127.30.194.3:41689)
I20260808 17:04:10.344358 21099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.340984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:55631 (local address 127.30.194.3:41689)
0808 17:04:10.341026 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:10.341027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:10.341142 (+ 115us) server_negotiation.cc:407] Connection header received
0808 17:04:10.341227 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:10.341228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:10.341238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:10.341272 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:10.341466 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.342142 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.343494 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.343988 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.344032 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.344040 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:10.344244 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 17:04:10.344294 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:04:10.344632 21100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.340635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:52621 (local address 127.30.194.1:36305)
0808 17:04:10.340680 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:10.340682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:10.340819 (+ 137us) server_negotiation.cc:407] Connection header received
0808 17:04:10.340943 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:10.340945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:10.340956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:10.341028 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:10.341227 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.341987 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.343551 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.344217 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.344267 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.344275 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:10.344517 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 17:04:10.344556 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:04:10.344738 21119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.340768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:41689 (local address 127.30.194.2:55631)
0808 17:04:10.341070 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:10.341078 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:10.341163 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:10.341328 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:10.341329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:10.341330 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:10.341401 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:10.341402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.342231 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.342232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:10.343398 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:10.343401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.344112 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.344113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:10.344175 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.344179 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:10.344604 (+ 425us) client_negotiation.cc:240] Negotiation successful
0808 17:04:10.344646 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":1,"thread_start_us":74,"threads_started":1}
I20260808 17:04:10.345126 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.340526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:36305 (local address 127.30.194.2:52621)
0808 17:04:10.340755 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:10.340767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:10.340844 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:10.341036 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:10.341037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:10.341040 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:10.341137 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:10.341140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.342092 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.342093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:10.343440 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:10.343443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.344336 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.344337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:10.344437 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.344441 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:10.344976 (+ 535us) client_negotiation.cc:240] Negotiation successful
0808 17:04:10.345020 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":24,"thread_start_us":119,"threads_started":1}
I20260808 17:04:10.345052 20956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "115f570887bc49a19a28477595c9b56a" candidate_uuid: "d4dbc0280d55411d9e1b444fc007614d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" is_pre_election: true
I20260808 17:04:10.345237 20956 raft_consensus.cc:2454] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4dbc0280d55411d9e1b444fc007614d in term 0.
I20260808 17:04:10.345429 20553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "115f570887bc49a19a28477595c9b56a" candidate_uuid: "d4dbc0280d55411d9e1b444fc007614d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de09fe68915482a8990468a04386b34" is_pre_election: true
I20260808 17:04:10.345573 20553 raft_consensus.cc:2454] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4dbc0280d55411d9e1b444fc007614d in term 0.
I20260808 17:04:10.345745 20666 leader_election.cc:302] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fecc9b5876e4ebcb129ef78d46c6cc7, d4dbc0280d55411d9e1b444fc007614d; no voters:
I20260808 17:04:10.345949 21115 raft_consensus.cc:2781] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:04:10.345996 21115 raft_consensus.cc:482] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:04:10.346019 21115 raft_consensus.cc:3037] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:10.346685 21115 raft_consensus.cc:504] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 } } peers { permanent_uuid: "d4dbc0280d55411d9e1b444fc007614d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40317 } } peers { permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 } }
I20260808 17:04:10.346797 21115 leader_election.cc:288] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [CANDIDATE]: Term 1 election: Requested vote from peers 8de09fe68915482a8990468a04386b34 (127.30.194.1:36305), 6fecc9b5876e4ebcb129ef78d46c6cc7 (127.30.194.3:41689)
I20260808 17:04:10.347020 20553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "115f570887bc49a19a28477595c9b56a" candidate_uuid: "d4dbc0280d55411d9e1b444fc007614d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de09fe68915482a8990468a04386b34"
I20260808 17:04:10.347021 20956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "115f570887bc49a19a28477595c9b56a" candidate_uuid: "d4dbc0280d55411d9e1b444fc007614d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7"
I20260808 17:04:10.347090 20553 raft_consensus.cc:3037] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:10.347090 20956 raft_consensus.cc:3037] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:10.347905 20956 raft_consensus.cc:2454] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbc0280d55411d9e1b444fc007614d in term 1.
I20260808 17:04:10.347913 20553 raft_consensus.cc:2454] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbc0280d55411d9e1b444fc007614d in term 1.
I20260808 17:04:10.348074 20666 leader_election.cc:302] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8de09fe68915482a8990468a04386b34, d4dbc0280d55411d9e1b444fc007614d; no voters:
I20260808 17:04:10.348170 21115 raft_consensus.cc:2781] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:04:10.348304 21115 raft_consensus.cc:686] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 1 LEADER]: Becoming Leader. State: Replica: d4dbc0280d55411d9e1b444fc007614d, State: Running, Role: LEADER
I20260808 17:04:10.348436 21115 consensus_queue.cc:237] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 } } peers { permanent_uuid: "d4dbc0280d55411d9e1b444fc007614d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40317 } } peers { permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 } }
I20260808 17:04:10.349259 20286 catalog_manager.cc:5495] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d reported cstate change: term changed from 0 to 1, leader changed from <none> to d4dbc0280d55411d9e1b444fc007614d (127.30.194.2). New cstate: current_term: 1 leader_uuid: "d4dbc0280d55411d9e1b444fc007614d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbc0280d55411d9e1b444fc007614d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 } health_report { overall_health: UNKNOWN } } }
Aug 08 17:04:10 dist-test-slave-2j7r krb5kdc[18614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208649, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for HTTP/127.30.194.20@KRBTEST.COM
I20260808 17:04:10.372866 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.369612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38466 (local address 127.30.194.2:40317)
0808 17:04:10.369653 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:10.369654 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:10.369661 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:04:10.369698 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:10.369698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:10.369718 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:10.369777 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:04:10.369955 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.370695 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.371797 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:10.372068 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:10.372123 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:10.372336 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:04:10.372463 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:04:10.372547 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:10.372769 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 17:04:10.372790 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:04:10.375057 20956 raft_consensus.cc:1261] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7 [term 1 FOLLOWER]: Refusing update from remote peer d4dbc0280d55411d9e1b444fc007614d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:10.375394 21115 consensus_queue.cc:1035] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fecc9b5876e4ebcb129ef78d46c6cc7" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 41689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:10.377723 20553 raft_consensus.cc:1261] T 115f570887bc49a19a28477595c9b56a P 8de09fe68915482a8990468a04386b34 [term 1 FOLLOWER]: Refusing update from remote peer d4dbc0280d55411d9e1b444fc007614d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:10.377988 21115 consensus_queue.cc:1035] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8de09fe68915482a8990468a04386b34" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:10.383329 21125 mvcc.cc:205] Tried to move back new op lower bound from 7316310631933128704 to 7316310631827685376. Current Snapshot: MvccSnapshot[applied={T|T < 7316310631933128704}]
I20260808 17:04:10.385921 21129 mvcc.cc:205] Tried to move back new op lower bound from 7316310631933128704 to 7316310631827685376. Current Snapshot: MvccSnapshot[applied={T|T < 7316310631933128704}]
I20260808 17:04:10.388036 20286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:04:10.388126 20286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:04:10.388881 20286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e5cd1473be1465da299fb68631f64a5: Sending DeleteTablet for 3 replicas of tablet 115f570887bc49a19a28477595c9b56a
I20260808 17:04:10.389273 20707 tablet_service.cc:1514] Processing DeleteTablet for tablet 115f570887bc49a19a28477595c9b56a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:10 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:38454
I20260808 17:04:10.389279 20511 tablet_service.cc:1514] Processing DeleteTablet for tablet 115f570887bc49a19a28477595c9b56a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:10 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:41492
I20260808 17:04:10.389281 20949 tablet_service.cc:1514] Processing DeleteTablet for tablet 115f570887bc49a19a28477595c9b56a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:10 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:59216
I20260808 17:04:10.389918 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 20343
I20260808 17:04:10.401371 21144 tablet_replica.cc:330] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: stopping tablet replica
I20260808 17:04:10.401484 21144 raft_consensus.cc:2229] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:04:10.401611 21144 raft_consensus.cc:2258] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:10.410969 21144 ts_tablet_manager.cc:1905] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:10.412570 21144 ts_tablet_manager.cc:1918] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:10.412714 21144 log.cc:1196] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/115f570887bc49a19a28477595c9b56a
I20260808 17:04:10.413062 21144 ts_tablet_manager.cc:1939] T 115f570887bc49a19a28477595c9b56a P d4dbc0280d55411d9e1b444fc007614d: Deleting consensus metadata
I20260808 17:04:10.413621 20271 catalog_manager.cc:4841] TS d4dbc0280d55411d9e1b444fc007614d (127.30.194.2:40317): tablet 115f570887bc49a19a28477595c9b56a (table test-table [id=674d56eabc29464289c10254c2a3262e]) successfully deleted
W20260808 17:04:10.415392 20271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:36305 (error 108)
I20260808 17:04:10.419431 21150 tablet_replica.cc:330] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: stopping tablet replica
I20260808 17:04:10.419559 21150 raft_consensus.cc:2229] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:10.419683 21150 raft_consensus.cc:2258] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 17:04:10.420079 21098 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:04:10.415758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:36305 (local address 127.0.0.1:41496)
0808 17:04:10.415808 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:10.415819 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:10.415854 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:10.420030 (+ 4176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:36305: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 17:04:10.420231 20271 catalog_manager.cc:4584] TS 8de09fe68915482a8990468a04386b34 (127.30.194.1:36305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 115f570887bc49a19a28477595c9b56a: Network error: Client connection negotiation failed: client connection to 127.30.194.1:36305: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 17:04:10.420531 21150 ts_tablet_manager.cc:1905] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:10.421833 21150 ts_tablet_manager.cc:1918] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:10.422444 21150 log.cc:1196] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/115f570887bc49a19a28477595c9b56a
I20260808 17:04:10.422887 21150 ts_tablet_manager.cc:1939] T 115f570887bc49a19a28477595c9b56a P 6fecc9b5876e4ebcb129ef78d46c6cc7: Deleting consensus metadata
I20260808 17:04:10.423368 20271 catalog_manager.cc:4841] TS 6fecc9b5876e4ebcb129ef78d46c6cc7 (127.30.194.3:41689): tablet 115f570887bc49a19a28477595c9b56a (table test-table [id=674d56eabc29464289c10254c2a3262e]) successfully deleted
I20260808 17:04:10.425060 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 20636
I20260808 17:04:10.444414 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 20849
I20260808 17:04:10.491514 21098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.491349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:36305 (local address 127.0.0.1:41512)
0808 17:04:10.491409 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:10.491464 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:36305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:04:10.491905 21098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:10.491807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:36305 (local address 127.0.0.1:41516)
0808 17:04:10.491849 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:10.491872 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:36305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 17:04:10.495667 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 20235
I20260808 17:04:10.510380 31496 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 17:04:10.863727 31496 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 17:04:10.863833 31496 mini_ranger.cc:66] Stopping Ranger...
I20260808 17:04:11.206962 31496 mini_ranger.cc:68] Stopped Ranger
2026-08-08 17:04:11.207 UTC [18676] LOG: received smart shutdown request
2026-08-08 17:04:11.211 UTC [18676] LOG: background worker "logical replication launcher" (PID 18687) exited with exit code 1
2026-08-08 17:04:11.211 UTC [18680] LOG: shutting down
2026-08-08 17:04:11.341 UTC [18676] LOG: database system is shut down
2026-08-08T17:04:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47076 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:04:11 dist-test-slave-2j7r krb5kdc[21166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:04:11 dist-test-slave-2j7r krb5kdc[21166](info): set up 2 sockets
Aug 08 17:04:11 dist-test-slave-2j7r krb5kdc[21166](info): commencing operation
krb5kdc: starting...
W20260808 17:04:12.568871 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:04:12 dist-test-slave-2j7r krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:04:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:04:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:04:12.677181 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:35495
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:35495,127.30.194.61:44661,127.30.194.60:45499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:12.755724 21186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:12.755908 21186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:12.755929 21186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:12.757014 21186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:12.757051 21186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:12.757064 21186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:12.757112 21186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:12.757138 21186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:12.758590 21186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:35495,127.30.194.61:44661,127.30.194.60:45499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:35495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:12.758802 21186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:12.759011 21186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:12.763490 21197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:12.763486 21195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:12.763485 21201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:12.763733 21186 server_base.cc:1034] running on GCE node
I20260808 17:04:12.763947 21186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:12.764230 21186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:12.765370 21186 hybrid_clock.cc:648] HybridClock initialized: now 1786208652765356 us; error 31 us; skew 500 ppm
Aug 08 17:04:12 dist-test-slave-2j7r krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:12.772445 21186 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:04:12.772818 21186 webserver.cc:460] Webserver started at http://127.30.194.62:38739/ using document root <none> and password file <none>
I20260808 17:04:12.773003 21186 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:12.773042 21186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:12.773166 21186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:12.774082 21186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "5a97e8fdc8294f30a5950885e633ba70"
format_stamp: "Formatted at 2026-08-08 17:04:12 on dist-test-slave-2j7r"
server_key: "3bfff311328375eb96a84b0b620b0450"
server_key_iv: "eb4fd93d875ba0b87908ca30da7f81d8"
server_key_version: "encryptionkey@0"
I20260808 17:04:12.774369 21186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "5a97e8fdc8294f30a5950885e633ba70"
format_stamp: "Formatted at 2026-08-08 17:04:12 on dist-test-slave-2j7r"
server_key: "3bfff311328375eb96a84b0b620b0450"
server_key_iv: "eb4fd93d875ba0b87908ca30da7f81d8"
server_key_version: "encryptionkey@0"
I20260808 17:04:12.775471 21186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:04:12.776063 21216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:12.776197 21186 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:12.776298 21186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "5a97e8fdc8294f30a5950885e633ba70"
format_stamp: "Formatted at 2026-08-08 17:04:12 on dist-test-slave-2j7r"
server_key: "3bfff311328375eb96a84b0b620b0450"
server_key_iv: "eb4fd93d875ba0b87908ca30da7f81d8"
server_key_version: "encryptionkey@0"
I20260808 17:04:12.776355 21186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:12.792827 21186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:12.794828 21186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:12.794960 21186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:12.798710 21186 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:35495
I20260808 17:04:12.798745 21296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:35495 every 8 connection(s)
I20260808 17:04:12.799039 21186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:04:12.799849 21299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:12.801160 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 21186
I20260808 17:04:12.801272 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:04:12.801483 31496 external_mini_cluster.cc:1428] Setting key 11d5d93b18a95fc1bc82612148212e7a
I20260808 17:04:12.801476 21299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:04:12 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.194.61@KRBTEST.COM; defaulting to no policy
I20260808 17:04:12.810971 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.802200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56726 (local address 127.30.194.62:35495)
0808 17:04:12.802366 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:12.802384 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:12.802449 (+ 65us) server_negotiation.cc:407] Connection header received
0808 17:04:12.802949 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:12.802951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:12.802971 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:12.803068 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:12.803741 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.804518 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.805725 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.806054 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.806137 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:12.808167 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:12.808174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:12.808175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:12.808181 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:12.809574 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:12.809900 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:12.809903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:12.809903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:12.809976 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:12.810167 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:12.810168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:12.810168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:12.810353 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:12.810471 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:12.810641 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 17:04:12.810745 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260808 17:04:12.810971 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.802177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35495 (local address 127.0.0.1:56726)
0808 17:04:12.802341 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.802378 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:12.802808 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:12.803094 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:12.803097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:12.803438 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:12.803653 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:12.803659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.804612 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.804613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:12.805649 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:12.805653 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.806137 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.806139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:12.806217 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:12.806643 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:12.806653 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:12.808087 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:12.809649 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:12.809655 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:12.809657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:12.809832 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:12.810035 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:12.810036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:12.810036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:12.810113 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:12.810462 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:12.810467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:12.810531 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 17:04:12.810608 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 17:04:12.810816 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":82,"thread_start_us":69,"threads_started":1}
I20260808 17:04:12.811658 21299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:04:12.811978 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.811861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:44661 (local address 127.0.0.1:53288)
0808 17:04:12.811904 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.811950 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.61@KRBTEST.COM" created.
W20260808 17:04:12.812115 21217 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.194.61:44661: connect: Connection refused (error 111)
I20260808 17:04:12.812500 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.812413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:44661 (local address 127.0.0.1:53290)
0808 17:04:12.812452 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.812478 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 17:04:12.812608 21299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.61:44661: Network error: Client connection negotiation failed: client connection to 127.30.194.61:44661: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab.
Entry for principal kudu/127.30.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab.
I20260808 17:04:12.825605 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.61:44661
--webserver_interface=127.30.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab
--principal=kudu/127.30.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:35495,127.30.194.61:44661,127.30.194.60:45499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:12.861819 21299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } attempt: 1
I20260808 17:04:12.862418 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.862265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:44661 (local address 127.0.0.1:53294)
0808 17:04:12.862334 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.862371 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 17:04:12.862888 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.862754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:44661 (local address 127.0.0.1:53308)
0808 17:04:12.862823 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.862859 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 17:04:12.862998 21299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.61:44661: Network error: Client connection negotiation failed: client connection to 127.30.194.61:44661: connect: Connection refused (error 111)
W20260808 17:04:12.907795 21314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:12.907974 21314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:12.908004 21314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:12.909118 21314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:12.909162 21314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:12.909181 21314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:12.909197 21314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:12.909219 21314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:12.910733 21314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:35495,127.30.194.61:44661,127.30.194.60:45499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab
--principal=kudu/127.30.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.61:44661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:12.910964 21314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:12.911166 21314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:12.915482 21325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:12.915500 21328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:12.915547 21330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:12.915817 21314 server_base.cc:1034] running on GCE node
I20260808 17:04:12.916033 21314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:12.916311 21314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:12.917441 21314 hybrid_clock.cc:648] HybridClock initialized: now 1786208652917428 us; error 31 us; skew 500 ppm
Aug 08 17:04:12 dist-test-slave-2j7r krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:12.919705 21314 init.cc:377] Logged in from keytab as kudu/127.30.194.61@KRBTEST.COM (short username kudu)
I20260808 17:04:12.920109 21314 webserver.cc:460] Webserver started at http://127.30.194.61:33971/ using document root <none> and password file <none>
I20260808 17:04:12.920312 21314 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:12.920358 21314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:12.920487 21314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:12.921479 21314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/data/instance:
uuid: "04a471732820427f92d18c1910792466"
format_stamp: "Formatted at 2026-08-08 17:04:12 on dist-test-slave-2j7r"
server_key: "8765448b819bc4c871e353cc679fb6c0"
server_key_iv: "fabd9bb504a1cf8055eb7db506f93fde"
server_key_version: "encryptionkey@0"
I20260808 17:04:12.921782 21314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/wal/instance:
uuid: "04a471732820427f92d18c1910792466"
format_stamp: "Formatted at 2026-08-08 17:04:12 on dist-test-slave-2j7r"
server_key: "8765448b819bc4c871e353cc679fb6c0"
server_key_iv: "fabd9bb504a1cf8055eb7db506f93fde"
server_key_version: "encryptionkey@0"
I20260808 17:04:12.923041 21314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:04:12.923697 21340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:12.923846 21314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:12.923928 21314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/wal
uuid: "04a471732820427f92d18c1910792466"
format_stamp: "Formatted at 2026-08-08 17:04:12 on dist-test-slave-2j7r"
server_key: "8765448b819bc4c871e353cc679fb6c0"
server_key_iv: "fabd9bb504a1cf8055eb7db506f93fde"
server_key_version: "encryptionkey@0"
I20260808 17:04:12.923983 21314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:12.931131 21299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } attempt: 2
I20260808 17:04:12.931452 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.931354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:44661 (local address 127.0.0.1:53318)
0808 17:04:12.931401 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.931426 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 17:04:12.931723 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.931649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:44661 (local address 127.0.0.1:53328)
0808 17:04:12.931681 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.931704 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 17:04:12.932029 21299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.61:44661: Network error: Client connection negotiation failed: client connection to 127.30.194.61:44661: connect: Connection refused (error 111)
I20260808 17:04:12.932940 21314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:12.934520 21314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:12.934643 21314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:12.938277 21314 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.61:44661
I20260808 17:04:12.938305 21416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.61:44661 every 8 connection(s)
I20260808 17:04:12.938593 21314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
I20260808 17:04:12.939348 21418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:12.939849 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 21314
I20260808 17:04:12.939951 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-1/wal/instance
I20260808 17:04:12.940140 31496 external_mini_cluster.cc:1428] Setting key ad4f6ea1abb1eee25bc979e64db59cea
I20260808 17:04:12.941080 21418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:04:12 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.194.60@KRBTEST.COM; defaulting to no policy
I20260808 17:04:12.949805 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.941762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.30.194.62:35495)
0808 17:04:12.941808 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:12.941810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:12.942024 (+ 214us) server_negotiation.cc:407] Connection header received
0808 17:04:12.942424 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:12.942426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:12.942440 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:12.942504 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:12.943022 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.943503 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.944879 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.945073 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.945150 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:12.947279 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:12.947280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:12.947281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:12.947288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:12.948552 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:12.948902 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:12.948903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:12.948903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:12.948960 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:12.949233 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:12.949234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:12.949234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:12.949339 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:12.949403 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:12.949643 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 17:04:12.949724 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:04:12.949956 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.941743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35495 (local address 127.0.0.1:56734)
0808 17:04:12.941901 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.941945 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:12.942347 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:12.942550 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:12.942553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:12.942770 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:12.942941 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:12.942944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.943592 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.943593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:12.944802 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:12.944806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.945165 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.945166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:12.945325 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:12.945716 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:12.945725 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:12.947193 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:12.948622 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:12.948629 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:12.948631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:12.948835 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:12.949014 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:12.949015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:12.949015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:12.949141 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:12.949397 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:12.949404 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:12.949530 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 17:04:12.949614 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 17:04:12.949753 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260808 17:04:12.950613 21418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.60@KRBTEST.COM" created.
Aug 08 17:04:12 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
I20260808 17:04:12.957376 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.950848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:44661 (local address 127.0.0.1:53334)
0808 17:04:12.950897 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.950907 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:12.950944 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:12.951186 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:12.951187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:12.951310 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:12.951370 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:12.951372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.952160 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.952161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:12.952699 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:12.952701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.953310 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.953311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:12.953364 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:12.953703 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:12.953707 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:12.954792 (+ 1085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:12.956495 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:12.956496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:12.956496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:12.956643 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:12.956834 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:12.956834 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:12.956835 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:12.956884 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:12.957177 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:12.957178 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:12.957210 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 17:04:12.957258 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 17:04:12.957320 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 17:04:12.957495 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.950883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.30.194.61:44661)
0808 17:04:12.951041 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:12.951045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:12.951068 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:04:12.951088 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:12.951090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:12.951103 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:12.951185 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:12.951449 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.952079 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.952786 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:12.953242 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:12.953312 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:12.954871 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:12.954888 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:12.954889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:12.954896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:12.956416 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:12.956713 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:12.956716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:12.956717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:12.956777 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:12.956938 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:12.956939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:12.956939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:12.957120 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:12.957215 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:12.957292 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 17:04:12.957377 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260808 17:04:12.958106 21418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:04:12.958384 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.958291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41658)
0808 17:04:12.958329 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.958366 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 17:04:12.958514 21345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
I20260808 17:04:12.958814 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:12.958757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41674)
0808 17:04:12.958778 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:12.958799 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260808 17:04:12.958915 21418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:45499: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab.
Entry for principal kudu/127.30.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab.
I20260808 17:04:12.965286 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.60:45499
--webserver_interface=127.30.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab
--principal=kudu/127.30.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:34003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:35495,127.30.194.61:44661,127.30.194.60:45499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:13.008113 21418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } attempt: 1
I20260808 17:04:13.008553 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.008401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41684)
0808 17:04:13.008474 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.008517 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 17:04:13.008886 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.008803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41690)
0808 17:04:13.008836 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.008860 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 17:04:13.008975 21418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:45499: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
I20260808 17:04:13.023180 21299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } attempt: 3
Aug 08 17:04:13 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
I20260808 17:04:13.030486 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.023577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.30.194.61:44661)
0808 17:04:13.023641 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.023643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.023655 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:04:13.023699 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.023700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.023711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.023765 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.024076 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.024842 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.025592 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.025910 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.025995 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.028062 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.028063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.028064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.028070 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.029519 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.029877 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.029877 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.029878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.029926 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.030099 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.030099 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.030099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.030188 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.030234 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.030373 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.030429 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:04:13.030546 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.023510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:44661 (local address 127.0.0.1:53336)
0808 17:04:13.023577 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.023589 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:13.023649 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:13.023786 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:13.023787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:13.023938 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:13.024008 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.024010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.024954 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.024955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.025519 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.025522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.025994 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.025995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.026078 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.026501 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:13.026505 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:13.027981 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:13.029597 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.029599 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.029599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.029808 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.029977 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.029978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.029978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.030045 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.030238 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:13.030239 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:13.030289 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 17:04:13.030356 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 17:04:13.030455 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 17:04:13.030972 21299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:04:13.031286 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.031181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41696)
0808 17:04:13.031226 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.031259 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 17:04:13.031590 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.031506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41702)
0808 17:04:13.031542 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.031569 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 17:04:13.031677 21299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:45499: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
W20260808 17:04:13.045421 21427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:13.045585 21427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:13.045606 21427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:13.046708 21427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:13.046746 21427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:13.046757 21427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:13.046782 21427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:13.046803 21427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:13.048266 21427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:35495,127.30.194.61:44661,127.30.194.60:45499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab
--principal=kudu/127.30.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.60:45499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:13.048493 21427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:13.048707 21427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:13.052994 21441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:13.052994 21439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:13.052989 21445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:13.053285 21427 server_base.cc:1034] running on GCE node
I20260808 17:04:13.053491 21427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:13.053723 21427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:13.054857 21427 hybrid_clock.cc:648] HybridClock initialized: now 1786208653054844 us; error 26 us; skew 500 ppm
Aug 08 17:04:13 dist-test-slave-2j7r krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208653, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:13.057008 21427 init.cc:377] Logged in from keytab as kudu/127.30.194.60@KRBTEST.COM (short username kudu)
I20260808 17:04:13.057438 21427 webserver.cc:460] Webserver started at http://127.30.194.60:40769/ using document root <none> and password file <none>
I20260808 17:04:13.057641 21427 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:13.057686 21427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:13.057803 21427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:13.058748 21427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/data/instance:
uuid: "6f8a9f20595e4eea8d273d3fa98386f8"
format_stamp: "Formatted at 2026-08-08 17:04:13 on dist-test-slave-2j7r"
server_key: "3f5fce4e53636893bd4434fdc4247d69"
server_key_iv: "de396b02a914a543275aa41f29fdc775"
server_key_version: "encryptionkey@0"
I20260808 17:04:13.059067 21427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/wal/instance:
uuid: "6f8a9f20595e4eea8d273d3fa98386f8"
format_stamp: "Formatted at 2026-08-08 17:04:13 on dist-test-slave-2j7r"
server_key: "3f5fce4e53636893bd4434fdc4247d69"
server_key_iv: "de396b02a914a543275aa41f29fdc775"
server_key_version: "encryptionkey@0"
I20260808 17:04:13.060220 21427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:04:13.060813 21453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:13.060992 21427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:13.061075 21427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/wal
uuid: "6f8a9f20595e4eea8d273d3fa98386f8"
format_stamp: "Formatted at 2026-08-08 17:04:13 on dist-test-slave-2j7r"
server_key: "3f5fce4e53636893bd4434fdc4247d69"
server_key_iv: "de396b02a914a543275aa41f29fdc775"
server_key_version: "encryptionkey@0"
I20260808 17:04:13.061161 21427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:13.062815 21299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } attempt: 1
I20260808 17:04:13.063113 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.063015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41704)
0808 17:04:13.063059 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.063086 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 17:04:13.063405 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.063313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41712)
0808 17:04:13.063350 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.063385 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 17:04:13.063486 21299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:45499: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
I20260808 17:04:13.077118 21418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } attempt: 2
I20260808 17:04:13.077443 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.077319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41714)
0808 17:04:13.077376 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.077412 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:04:13.077725 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.077657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41720)
0808 17:04:13.077690 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.077710 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 17:04:13.077807 21418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:45499: Network error: Client connection negotiation failed: client connection to 127.30.194.60:45499: connect: Connection refused (error 111)
I20260808 17:04:13.081187 21427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:13.082679 21427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:13.082793 21427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:13.086424 21427 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.60:45499
I20260808 17:04:13.086453 21550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.60:45499 every 8 connection(s)
I20260808 17:04:13.086740 21427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
I20260808 17:04:13.087496 21551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:13.088850 21551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:04:13.089447 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 21427
I20260808 17:04:13.089550 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208501350379-31496-0/minicluster-data/master-2/wal/instance
I20260808 17:04:13.089721 31496 external_mini_cluster.cc:1428] Setting key 1575e464794942b9976e1ed7ee0e5743
Aug 08 17:04:13 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208653, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
Aug 08 17:04:13 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
I20260808 17:04:13.097572 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.089465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.30.194.62:35495)
0808 17:04:13.089521 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.089523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.089713 (+ 190us) server_negotiation.cc:407] Connection header received
0808 17:04:13.090264 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.090266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.090277 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.090326 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.090836 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.091287 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.092207 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.092410 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.092465 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.094542 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.094544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.094544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.094551 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.096482 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.096779 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.096779 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.096780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.096828 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.097031 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.097031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.097032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.097183 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.097231 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.097410 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.097488 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:04:13.097755 21554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.089450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:35495 (local address 127.0.0.1:56748)
0808 17:04:13.089593 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.089630 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:13.090177 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:13.090384 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:13.090389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:13.090616 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:13.090751 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.090755 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.091355 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.091356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.092128 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.092130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.092479 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.092480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.092607 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.093008 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:13.093018 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:13.094464 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:13.096555 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.096558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.096560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.096714 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.096884 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.096885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.096885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.096974 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.097235 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:13.097240 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:13.097293 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 17:04:13.097394 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 17:04:13.097547 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":77,"thread_start_us":71,"threads_started":1}
I20260808 17:04:13.097813 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.090067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.30.194.60:45499)
0808 17:04:13.090225 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.090229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.090292 (+ 63us) server_negotiation.cc:407] Connection header received
0808 17:04:13.090383 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.090385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.090406 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.090482 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.091017 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.091771 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.092441 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.092731 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.092786 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.095105 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.095111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.095111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.095117 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.096390 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.096746 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.096748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.096749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.096835 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.097019 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.097019 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.097020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.097268 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.097410 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.097478 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.097593 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260808 17:04:13.098359 21551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:04:13 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208653, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
Aug 08 17:04:13 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:13.104091 21554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.098563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:44661 (local address 127.0.0.1:53348)
0808 17:04:13.098618 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.098626 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:13.098658 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:13.098767 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:13.098768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:13.098891 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:13.098937 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.098938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.099502 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.099503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.099955 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.099957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.100274 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.100275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.100337 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.100663 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:13.100667 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:13.101755 (+ 1088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:13.103240 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.103240 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.103241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.103404 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.103591 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.103591 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.103591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.103639 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.103834 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:13.103834 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:13.103874 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 17:04:13.103936 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 17:04:13.104009 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 17:04:13.104106 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.098607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.30.194.61:44661)
0808 17:04:13.098643 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.098645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.098652 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:04:13.098713 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.098714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.098723 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.098766 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.099005 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.099440 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.100025 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.100220 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.100272 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.101831 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.101833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.101833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.101839 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.103169 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.103472 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.103473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.103473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.103533 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.103687 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.103687 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.103688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.103782 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.103829 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.103971 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.104042 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:04:13.104429 21551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:04:13.105027 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.098545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56756 (local address 127.30.194.62:35495)
0808 17:04:13.098577 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.098579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.098587 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:04:13.098635 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.098635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.098643 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.098686 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.099090 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.099511 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.100120 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.100308 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.100359 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.102417 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.102418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.102418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.102425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.103587 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.103887 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.103888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.103888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.103937 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.104116 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.104116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.104117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.104613 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.104777 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.104877 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.104944 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 17:04:13 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208653, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
Aug 08 17:04:13 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
I20260808 17:04:13.111148 21554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.104632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41736)
0808 17:04:13.105030 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.105041 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:13.105197 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:13.105306 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:13.105306 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:13.105423 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:13.105449 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.105450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.106243 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.106244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.106955 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.106957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.107377 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.107377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.107408 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.107719 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:13.107723 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:13.108867 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:13.110215 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.110216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.110217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.110410 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.110581 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.110582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.110582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.110636 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.110841 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:13.110841 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:13.110893 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 17:04:13.110957 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:04:13.111044 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383}
I20260808 17:04:13.111161 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.104686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.30.194.60:45499)
0808 17:04:13.104721 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.104723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.105078 (+ 355us) server_negotiation.cc:407] Connection header received
0808 17:04:13.105247 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.105248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.105256 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.105301 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.105517 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.106175 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.107016 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.107304 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.107361 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.108958 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.108959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.108959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.108965 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.110137 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.110474 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.110475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.110475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.110521 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.110694 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.110696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.110696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.110785 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.110831 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.110984 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.111085 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":18}
I20260808 17:04:13.112287 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.105581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.30.194.61:44661)
0808 17:04:13.105615 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.105616 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.105624 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:04:13.105658 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.105659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.105667 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.105698 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.106200 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.106875 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.107629 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.107817 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.107867 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.109905 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.109906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.109906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.109913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.111307 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.111634 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.111635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.111636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.111695 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.111877 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.111877 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.111878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.111963 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.112059 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.112137 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.112199 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 17:04:13.112573 21551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8: Bootstrap starting.
I20260808 17:04:13.113238 21551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:13.113543 21551 log.cc:824] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:13.113854 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:04:13.114279 21551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8: No bootstrap required, opened a new log
I20260808 17:04:13.116189 21551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } }
I20260808 17:04:13.116331 21551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:13.116374 21551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8a9f20595e4eea8d273d3fa98386f8, State: Initialized, Role: FOLLOWER
I20260808 17:04:13.116506 21551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } }
I20260808 17:04:13.116784 21551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 17:04:13.118570 21589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:04:13.118621 21589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:04:13.120085 21574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } } }
I20260808 17:04:13.120175 21574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:04:13.118765 21427 master_runner.cc:186] Error getting master registration for 127.30.194.62:35495: OK, instance_id { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" instance_seqno: 1786208652797677 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:04:13.120798 21427 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:04:13.123451 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.115249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.30.194.62:35495)
0808 17:04:13.115283 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.115285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.115628 (+ 343us) server_negotiation.cc:407] Connection header received
0808 17:04:13.115692 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.115693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.115702 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.115750 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.116463 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.117154 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.118399 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.118674 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.118705 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.120154 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.120155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.120155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.120162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.121720 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.122564 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.122565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.122566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.122622 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.123022 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.123022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.123023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.123117 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.123194 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.123320 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.123398 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:04:13.124718 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.115464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.30.194.60:45499)
0808 17:04:13.115498 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.115500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.115763 (+ 263us) server_negotiation.cc:407] Connection header received
0808 17:04:13.115845 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.115846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.115865 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.115892 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.116726 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.117379 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.118380 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.118657 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.118693 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.121002 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.121004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.121004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.121012 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.122838 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.123989 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.123990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.123991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.124048 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.124232 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.124233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.124233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.124368 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.124443 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.124573 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.124637 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:04:13.124846 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.115261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.30.194.61:44661)
0808 17:04:13.115294 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.115296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.115564 (+ 268us) server_negotiation.cc:407] Connection header received
0808 17:04:13.115620 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.115621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.115631 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.115675 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.116459 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.117067 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.118127 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.118392 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.118436 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.122158 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.122159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.122160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.122166 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.123587 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.124028 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.124029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.124029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.124087 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.124274 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.124275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.124275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.124407 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.124503 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.124689 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.124766 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:04:13.138608 21299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } attempt: 2
Aug 08 17:04:13 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
I20260808 17:04:13.144511 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.138936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41754 (local address 127.30.194.60:45499)
0808 17:04:13.139006 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.139008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.139014 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:04:13.139031 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.139031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.139039 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.139086 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.139397 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.139795 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.140696 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.140854 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.140889 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.142528 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.142529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.142529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.142535 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.143641 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.143966 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.143966 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.143967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.144006 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.144157 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.144157 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.144158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.144236 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.144276 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.144414 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.144461 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 17:04:13.144619 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.138806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41754)
0808 17:04:13.138903 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.138913 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:13.138947 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:13.139127 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:13.139128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:13.139280 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:13.139317 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.139318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.139855 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.139856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.140630 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.140632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.140904 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.140905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.140957 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.141285 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:13.141289 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:13.142452 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:13.143709 (+ 1257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.143709 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.143710 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.143905 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.144053 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.144054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.144054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.144111 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.144280 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:13.144281 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:13.144321 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 17:04:13.144398 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 17:04:13.144481 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 17:04:13.145834 21299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70: Bootstrap starting.
I20260808 17:04:13.146478 21299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:13.146785 21299 log.cc:824] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:13.147454 21299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70: No bootstrap required, opened a new log
I20260808 17:04:13.148813 21299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } }
I20260808 17:04:13.148932 21299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:13.148954 21299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a97e8fdc8294f30a5950885e633ba70, State: Initialized, Role: FOLLOWER
I20260808 17:04:13.149062 21299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } }
I20260808 17:04:13.149315 21593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } } }
I20260808 17:04:13.149369 21299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:04:13.149408 21593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 17:04:13.150929 21606 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:04:13.150979 21606 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:04:13.151681 21186 master_runner.cc:186] Error getting master registration for 127.30.194.61:44661: OK, instance_id { permanent_uuid: "04a471732820427f92d18c1910792466" instance_seqno: 1786208652937256 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:04:13.154168 21186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:04:13.156734 21574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:13.156826 21574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } }
I20260808 17:04:13.157073 21574 leader_election.cc:288] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a97e8fdc8294f30a5950885e633ba70 (127.30.194.62:35495), 04a471732820427f92d18c1910792466 (127.30.194.61:44661)
I20260808 17:04:13.157585 21259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a97e8fdc8294f30a5950885e633ba70" is_pre_election: true
I20260808 17:04:13.157725 21259 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8a9f20595e4eea8d273d3fa98386f8 in term 0.
I20260808 17:04:13.157955 21461 leader_election.cc:302] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a97e8fdc8294f30a5950885e633ba70, 6f8a9f20595e4eea8d273d3fa98386f8; no voters:
I20260808 17:04:13.158077 21574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:04:13.158120 21574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:04:13.158147 21574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:13.158954 21574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } }
I20260808 17:04:13.159072 21574 leader_election.cc:288] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [CANDIDATE]: Term 1 election: Requested vote from peers 5a97e8fdc8294f30a5950885e633ba70 (127.30.194.62:35495), 04a471732820427f92d18c1910792466 (127.30.194.61:44661)
I20260808 17:04:13.159361 21259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a97e8fdc8294f30a5950885e633ba70"
I20260808 17:04:13.159435 21259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:13.160190 21259 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8a9f20595e4eea8d273d3fa98386f8 in term 1.
I20260808 17:04:13.160349 21461 leader_election.cc:302] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a97e8fdc8294f30a5950885e633ba70, 6f8a9f20595e4eea8d273d3fa98386f8; no voters:
I20260808 17:04:13.160434 21574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:04:13.160562 21574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8a9f20595e4eea8d273d3fa98386f8, State: Running, Role: LEADER
I20260808 17:04:13.160648 21574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } }
I20260808 17:04:13.161022 21609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f8a9f20595e4eea8d273d3fa98386f8. Latest consensus state: current_term: 1 leader_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } } }
I20260808 17:04:13.161173 21609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:13.161459 21615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:04:13.161801 21615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:04:13.164108 21259 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [term 1 FOLLOWER]: Refusing update from remote peer 6f8a9f20595e4eea8d273d3fa98386f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:13.164533 21609 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:13.165901 21593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f8a9f20595e4eea8d273d3fa98386f8. Latest consensus state: current_term: 1 leader_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } } }
I20260808 17:04:13.165990 21593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:04:13.167339 21609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } } }
I20260808 17:04:13.167421 21609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:13.167522 21593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } } }
I20260808 17:04:13.167601 21593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:04:13.167562 21616 mvcc.cc:205] Tried to move back new op lower bound from 7316310643357593600 to 7316310643346558976. Current Snapshot: MvccSnapshot[applied={T|T < 7316310643357593600}]
I20260808 17:04:13.167959 21618 mvcc.cc:205] Tried to move back new op lower bound from 7316310643357593600 to 7316310643346558976. Current Snapshot: MvccSnapshot[applied={T|T < 7316310643357593600}]
I20260808 17:04:13.168505 21615 catalog_manager.cc:1347] Generated new cluster ID: 4193a815baaf45f8b444a7395cb32739
I20260808 17:04:13.168545 21615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:04:13.173156 21418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } attempt: 3
I20260808 17:04:13.176100 21615 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:04:13.176761 21615 catalog_manager.cc:1504] Loading token signing keys...
Aug 08 17:04:13 dist-test-slave-2j7r krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208652, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
I20260808 17:04:13.182243 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.173447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:45499 (local address 127.0.0.1:41758)
0808 17:04:13.173562 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:13.173571 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:13.173608 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:13.174598 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:13.174599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:13.174768 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:13.174812 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.174815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.175954 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.175955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.176751 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:13.176754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.177382 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.177383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:13.177458 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.177859 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:13.177865 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:13.179381 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:13.181142 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.181143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.181144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.181352 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.181597 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:13.181597 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:13.181598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:13.181665 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:13.181954 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:13.181955 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:13.182010 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 17:04:13.182064 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:04:13.182152 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 17:04:13.182734 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:13.173671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41758 (local address 127.30.194.60:45499)
0808 17:04:13.173724 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:13.173726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:13.173732 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:04:13.173749 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:13.173750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:13.173758 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:13.173799 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:13.175120 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.175845 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.176917 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:13.177242 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:13.177299 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:13.179489 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:13.179491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:13.179491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:13.179497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:13.181008 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.181448 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.181450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.181450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.181520 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:13.181735 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:13.181736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:13.181736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:13.181862 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:13.181912 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:13.182562 (+ 650us) server_negotiation.cc:299] Negotiation successful
0808 17:04:13.182650 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:04:13.184057 21615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8: Generated new TSK 0
I20260808 17:04:13.184208 21615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:04:13.184315 21418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466: Bootstrap starting.
I20260808 17:04:13.184994 21418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:13.185297 21418 log.cc:824] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:13.186125 21418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466: No bootstrap required, opened a new log
I20260808 17:04:13.190042 21418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } }
I20260808 17:04:13.190184 21418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:13.190233 21418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a471732820427f92d18c1910792466, State: Initialized, Role: FOLLOWER
I20260808 17:04:13.190362 21418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } }
I20260808 17:04:13.190681 21418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:04:13.190776 21376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a471732820427f92d18c1910792466"
I20260808 17:04:13.190879 21376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:13.191658 21376 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8a9f20595e4eea8d273d3fa98386f8 in term 1.
I20260808 17:04:13.191771 21377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a471732820427f92d18c1910792466" is_pre_election: true
I20260808 17:04:13.191833 21377 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6f8a9f20595e4eea8d273d3fa98386f8 in term 1. Re-sending same reply.
I20260808 17:04:13.192255 21375 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [term 1 FOLLOWER]: Refusing update from remote peer 6f8a9f20595e4eea8d273d3fa98386f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:13.192580 21574 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 17:04:13.194536 21650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:04:13.194589 21650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:04:13.197634 21656 mvcc.cc:205] Tried to move back new op lower bound from 7316310643437535232 to 7316310643346558976. Current Snapshot: MvccSnapshot[applied={T|T < 7316310643357593600}]
I20260808 17:04:13.197729 21658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } } }
I20260808 17:04:13.197803 21658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:04:13.197849 21658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f8a9f20595e4eea8d273d3fa98386f8. Latest consensus state: current_term: 1 leader_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } } }
I20260808 17:04:13.197906 21658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:04:13.197937 21658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } } }
I20260808 17:04:13.197978 21658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:04:13.637409 21574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a97e8fdc8294f30a5950885e633ba70" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 35495 } } peers { permanent_uuid: "04a471732820427f92d18c1910792466" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 44661 } } peers { permanent_uuid: "6f8a9f20595e4eea8d273d3fa98386f8" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 45499 } } }
I20260808 17:04:13.637560 21574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8a9f20595e4eea8d273d3fa98386f8 [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:14.151439 21606 catalog_manager.cc:1259] Loaded cluster ID: 4193a815baaf45f8b444a7395cb32739
I20260808 17:04:14.151522 21606 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70: loading cluster ID for follower catalog manager: success
I20260808 17:04:14.152312 21606 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70: acquiring CA information for follower catalog manager: success
I20260808 17:04:14.152580 21606 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5a97e8fdc8294f30a5950885e633ba70: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:04:14.194988 21650 catalog_manager.cc:1259] Loaded cluster ID: 4193a815baaf45f8b444a7395cb32739
I20260808 17:04:14.195062 21650 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466: loading cluster ID for follower catalog manager: success
I20260808 17:04:14.195854 21650 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466: acquiring CA information for follower catalog manager: success
I20260808 17:04:14.196159 21650 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 04a471732820427f92d18c1910792466: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:04:14.242476 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 21186
I20260808 17:04:14.254148 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 21314
I20260808 17:04:14.264046 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 21427
2026-08-08T17:04:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2836 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:04:14 dist-test-slave-2j7r krb5kdc[21672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:04:14 dist-test-slave-2j7r krb5kdc[21672](info): set up 2 sockets
Aug 08 17:04:14 dist-test-slave-2j7r krb5kdc[21672](info): commencing operation
krb5kdc: starting...
W20260808 17:04:15.409986 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:04:15 dist-test-slave-2j7r krb5kdc[21672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:04:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:04:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:04:15.515864 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:42053
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:46877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:42053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:15.593798 21690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:15.593991 21690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:15.594019 21690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:15.595113 21690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:15.595151 21690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:15.595170 21690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:15.595184 21690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:15.595201 21690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:15.597070 21690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:46877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:42053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:42053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:15.597327 21690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:15.597515 21690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:15.601963 21698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:15.601975 21699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:15.601975 21701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:15.602190 21690 server_base.cc:1034] running on GCE node
I20260808 17:04:15.602396 21690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:15.602617 21690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:15.603745 21690 hybrid_clock.cc:648] HybridClock initialized: now 1786208655603732 us; error 27 us; skew 500 ppm
Aug 08 17:04:15 dist-test-slave-2j7r krb5kdc[21672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208655, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:15.606093 21690 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:04:15.606498 21690 webserver.cc:460] Webserver started at http://127.30.194.62:46409/ using document root <none> and password file <none>
I20260808 17:04:15.606681 21690 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:15.606721 21690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:15.606838 21690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:15.607857 21690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "0cf29a715d384e329b79a1a382051bff"
format_stamp: "Formatted at 2026-08-08 17:04:15 on dist-test-slave-2j7r"
server_key: "f0f723c3ef662de5dfca42a2b59624a7"
server_key_iv: "15a4e34d220d763a3ef5792e02d40a8d"
server_key_version: "encryptionkey@0"
I20260808 17:04:15.608247 21690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "0cf29a715d384e329b79a1a382051bff"
format_stamp: "Formatted at 2026-08-08 17:04:15 on dist-test-slave-2j7r"
server_key: "f0f723c3ef662de5dfca42a2b59624a7"
server_key_iv: "15a4e34d220d763a3ef5792e02d40a8d"
server_key_version: "encryptionkey@0"
I20260808 17:04:15.609776 21690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:04:15.610622 21714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:15.610786 21690 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:04:15.610877 21690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "0cf29a715d384e329b79a1a382051bff"
format_stamp: "Formatted at 2026-08-08 17:04:15 on dist-test-slave-2j7r"
server_key: "f0f723c3ef662de5dfca42a2b59624a7"
server_key_iv: "15a4e34d220d763a3ef5792e02d40a8d"
server_key_version: "encryptionkey@0"
I20260808 17:04:15.610945 21690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:15.634800 21690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:15.636740 21690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:15.636875 21690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:15.640504 21690 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:42053
I20260808 17:04:15.640535 21795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:42053 every 8 connection(s)
I20260808 17:04:15.640828 21690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:04:15.641639 21796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:15.643685 21796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff: Bootstrap starting.
I20260808 17:04:15.644191 21796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:15.644418 21796 log.cc:824] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:15.645015 21796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff: No bootstrap required, opened a new log
I20260808 17:04:15.646536 21796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf29a715d384e329b79a1a382051bff" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42053 } }
I20260808 17:04:15.646662 21796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:15.646682 21796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf29a715d384e329b79a1a382051bff, State: Initialized, Role: FOLLOWER
I20260808 17:04:15.646792 21796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf29a715d384e329b79a1a382051bff" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42053 } }
I20260808 17:04:15.646842 21796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:04:15.646859 21796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:04:15.646889 21796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:15.647331 21796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf29a715d384e329b79a1a382051bff" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42053 } }
I20260808 17:04:15.647411 21796 leader_election.cc:302] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cf29a715d384e329b79a1a382051bff; no voters:
I20260808 17:04:15.647535 21796 leader_election.cc:288] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:04:15.647574 21807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:04:15.647745 21807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [term 1 LEADER]: Becoming Leader. State: Replica: 0cf29a715d384e329b79a1a382051bff, State: Running, Role: LEADER
I20260808 17:04:15.647747 21796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:04:15.647841 21807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf29a715d384e329b79a1a382051bff" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42053 } }
I20260808 17:04:15.648200 21808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cf29a715d384e329b79a1a382051bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf29a715d384e329b79a1a382051bff" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42053 } } }
I20260808 17:04:15.648281 21808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:15.648384 21809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cf29a715d384e329b79a1a382051bff. Latest consensus state: current_term: 1 leader_uuid: "0cf29a715d384e329b79a1a382051bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf29a715d384e329b79a1a382051bff" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 42053 } } }
I20260808 17:04:15.648448 21809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:15.648865 21817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:04:15.649420 21817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:04:15.649765 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 21690
I20260808 17:04:15.649863 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:04:15.650063 31496 external_mini_cluster.cc:1428] Setting key dadd09e9c54c07cff5e068889fbc0e8d
I20260808 17:04:15.650802 21817 catalog_manager.cc:1347] Generated new cluster ID: b3076da6987e4ff2a16c7547585f1ef2
I20260808 17:04:15.650854 21817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:04:15 dist-test-slave-2j7r krb5kdc[21672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:15.660164 21817 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:04:15.660620 21817 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:04:15.661267 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:15.650546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59238 (local address 127.30.194.62:42053)
0808 17:04:15.650685 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:15.650694 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:15.650910 (+ 216us) server_negotiation.cc:407] Connection header received
0808 17:04:15.651022 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:15.651029 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:15.651079 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:15.651193 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:15.651907 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:15.653279 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:15.654341 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:15.654932 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:15.655021 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:15.657634 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:15.657639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:15.657640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:15.657652 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:15.659492 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:15.659890 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:15.659894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:15.659895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:15.660013 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:15.660274 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:15.660275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:15.660275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:15.660569 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:15.660786 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:15.660858 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 17:04:15.661004 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260808 17:04:15.662359 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:04:15.668697 21817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cf29a715d384e329b79a1a382051bff: Generated new TSK 0
I20260808 17:04:15.668846 21817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:04:15.715955 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 21690
2026-08-08T17:04:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1454 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:04:15 dist-test-slave-2j7r krb5kdc[21842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:04:15 dist-test-slave-2j7r krb5kdc[21842](info): set up 2 sockets
Aug 08 17:04:15 dist-test-slave-2j7r krb5kdc[21842](info): commencing operation
krb5kdc: starting...
W20260808 17:04:16.884591 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:04:16 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:04:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:04:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:04:16.996665 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40309
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:40309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:17.076442 21859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:17.076634 21859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:17.076654 21859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:17.077726 21859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:17.077759 21859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:17.077770 21859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:17.077781 21859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:17.077792 21859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:17.079170 21859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:40309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:17.079362 21859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:17.079528 21859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:17.084110 21870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:17.084120 21866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:17.084122 21867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:17.084373 21859 server_base.cc:1034] running on GCE node
I20260808 17:04:17.084630 21859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:17.084893 21859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:17.086031 21859 hybrid_clock.cc:648] HybridClock initialized: now 1786208657086021 us; error 27 us; skew 500 ppm
Aug 08 17:04:17 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208657, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:17.088370 21859 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:04:17.088800 21859 webserver.cc:460] Webserver started at http://127.30.194.62:39431/ using document root <none> and password file <none>
I20260808 17:04:17.088994 21859 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:17.089041 21859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:17.089183 21859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:17.090179 21859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "c7b0929cd4a74a77baf22d08a08b510b"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "5fbc5d291cbc82efb59e75abd22c783f"
server_key_iv: "5530db40daa64dfb490c08a9bb9b4fe1"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.090512 21859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "c7b0929cd4a74a77baf22d08a08b510b"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "5fbc5d291cbc82efb59e75abd22c783f"
server_key_iv: "5530db40daa64dfb490c08a9bb9b4fe1"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.091701 21859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:04:17.092319 21878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.092469 21859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:17.092551 21859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "c7b0929cd4a74a77baf22d08a08b510b"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "5fbc5d291cbc82efb59e75abd22c783f"
server_key_iv: "5530db40daa64dfb490c08a9bb9b4fe1"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.092609 21859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:17.131302 21859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:17.133152 21859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:17.133311 21859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:17.137400 21859 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:40309
I20260808 17:04:17.137435 21960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:40309 every 8 connection(s)
I20260808 17:04:17.137784 21859 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:04:17.138739 21962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:17.141026 21962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Bootstrap starting.
I20260808 17:04:17.141101 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 21859
I20260808 17:04:17.141201 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:04:17.141433 31496 external_mini_cluster.cc:1428] Setting key 759677033696a8c59fb45f81f8065215
I20260808 17:04:17.141744 21962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:17.142082 21962 log.cc:824] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:17.142777 21962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: No bootstrap required, opened a new log
I20260808 17:04:17.144855 21962 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:17.145040 21962 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:17.145076 21962 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b0929cd4a74a77baf22d08a08b510b, State: Initialized, Role: FOLLOWER
I20260808 17:04:17.145233 21962 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:17.145324 21962 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:04:17.145358 21962 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:04:17.145401 21962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:17.146000 21962 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:17.146091 21962 leader_election.cc:302] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7b0929cd4a74a77baf22d08a08b510b; no voters:
I20260808 17:04:17.146246 21962 leader_election.cc:288] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:04:17.146322 21971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:04:17.146490 21971 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 1 LEADER]: Becoming Leader. State: Replica: c7b0929cd4a74a77baf22d08a08b510b, State: Running, Role: LEADER
I20260808 17:04:17.146602 21971 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:17.146780 21962 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:04:17.148001 21972 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7b0929cd4a74a77baf22d08a08b510b. Latest consensus state: current_term: 1 leader_uuid: "c7b0929cd4a74a77baf22d08a08b510b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } } }
I20260808 17:04:17.148092 21972 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:17.148268 21973 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7b0929cd4a74a77baf22d08a08b510b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } } }
I20260808 17:04:17.148322 21973 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: This master's current role is: LEADER
Aug 08 17:04:17 dist-test-slave-2j7r krb5kdc[21842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:17.148458 21998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:04:17.149010 21998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:04:17.150568 21998 catalog_manager.cc:1347] Generated new cluster ID: 3063efba345d4478872cf3c91a939205
I20260808 17:04:17.150621 21998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:04:17.152350 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.141939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.30.194.62:40309)
0808 17:04:17.142154 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.142165 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.142202 (+ 37us) server_negotiation.cc:407] Connection header received
0808 17:04:17.142751 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.142760 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.142813 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.142954 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:17.143730 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.144633 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.145705 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.146171 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.146293 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.148949 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:17.148953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:17.148954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:17.148966 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:17.150851 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:17.151180 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:17.151183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:17.151183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:17.151278 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:17.151472 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:17.151473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:17.151473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:17.151732 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:17.151904 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.151998 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.152132 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":87,"threads_started":1}
I20260808 17:04:17.168161 21998 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:04:17.168676 21998 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:04:17.172255 21998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Generated new TSK 0
I20260808 17:04:17.172422 21998 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:04:17.203054 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40309
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:17.281412 22005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:17.281592 22005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:17.281621 22005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:17.282696 22005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:17.282735 22005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:17.282769 22005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:04:17.284148 22005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:17.284372 22005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:17.284564 22005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:17.289101 22018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:17.289139 22016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:17.289139 22014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:17.289340 22005 server_base.cc:1034] running on GCE node
I20260808 17:04:17.289525 22005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:17.289726 22005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:17.290871 22005 hybrid_clock.cc:648] HybridClock initialized: now 1786208657290862 us; error 33 us; skew 500 ppm
Aug 08 17:04:17 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208657, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:17.293124 22005 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:04:17.293555 22005 webserver.cc:460] Webserver started at http://127.30.194.1:34321/ using document root <none> and password file <none>
I20260808 17:04:17.293743 22005 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:17.293788 22005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:17.293911 22005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:17.294843 22005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "b7b2bfed18ce1a027a0991ed423f6c77"
server_key_iv: "e991e3390bc9e7a37826baac055c4444"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.295167 22005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "b7b2bfed18ce1a027a0991ed423f6c77"
server_key_iv: "e991e3390bc9e7a37826baac055c4444"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.296389 22005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:04:17.296998 22028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.297156 22005 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.297237 22005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "b7b2bfed18ce1a027a0991ed423f6c77"
server_key_iv: "e991e3390bc9e7a37826baac055c4444"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.297302 22005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:17.307996 22005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:17.309540 22005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:17.309659 22005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:17.309851 22005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:17.310110 22005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:17.310142 22005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.310168 22005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:17.310185 22005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.315516 22005 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:33709
I20260808 17:04:17.315550 22196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:33709 every 8 connection(s)
I20260808 17:04:17.315805 22005 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:04:17.317596 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 22005
I20260808 17:04:17.317682 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:04:17.317834 31496 external_mini_cluster.cc:1428] Setting key 9d9895c732e430285023bbc76815465d
Aug 08 17:04:17 dist-test-slave-2j7r krb5kdc[21842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208657, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:17.324317 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.316393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:49325 (local address 127.30.194.62:40309)
0808 17:04:17.316441 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.316444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.316610 (+ 166us) server_negotiation.cc:407] Connection header received
0808 17:04:17.317004 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.317005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.317013 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.317064 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:17.317936 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.318613 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.319529 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.319744 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.319797 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.321718 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:17.321719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:17.321720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:17.321727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:17.323106 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:17.323477 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:17.323478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:17.323479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:17.323542 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:17.323738 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:17.323738 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:17.323739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:17.323870 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:17.323941 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.324142 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.324225 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:04:17.324349 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.316361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:49325)
0808 17:04:17.316504 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:17.316541 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:17.316935 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:17.317126 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:17.317129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:17.317650 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:17.317848 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.317853 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.318696 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.318698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.319449 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.319451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.319812 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.319813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.319925 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.320337 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:17.320345 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:17.321633 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:17.323183 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:17.323189 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:17.323190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:17.323409 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:17.323601 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:17.323601 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:17.323601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:17.323686 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:17.323928 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:17.323931 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:17.324029 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 17:04:17.324101 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:04:17.324206 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260808 17:04:17.324772 22198 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40309
I20260808 17:04:17.324872 22198 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:17.325052 22198 heartbeater.cc:511] Master 127.30.194.62:40309 requested a full tablet report, sending...
I20260808 17:04:17.325546 21905 ts_manager.cc:194] Registered new tserver with Master: 289bfa4ac1fc4059b99ac51c06b7c9cd (127.30.194.1:33709)
I20260808 17:04:17.326141 21905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:49325
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:04:17.342905 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40309
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:17.422174 22207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:17.422356 22207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:17.422377 22207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:17.423471 22207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:17.423511 22207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:17.423538 22207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:04:17.424955 22207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:17.425189 22207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:17.425392 22207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:17.429961 22220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:17.429958 22219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:17.429966 22224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:17.430171 22207 server_base.cc:1034] running on GCE node
I20260808 17:04:17.430362 22207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:17.430540 22207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:17.431684 22207 hybrid_clock.cc:648] HybridClock initialized: now 1786208657431666 us; error 28 us; skew 500 ppm
Aug 08 17:04:17 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208657, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:17.434052 22207 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:04:17.434432 22207 webserver.cc:460] Webserver started at http://127.30.194.2:33147/ using document root <none> and password file <none>
I20260808 17:04:17.434625 22207 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:17.434669 22207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:17.434788 22207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:17.435724 22207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "287e98c805193f6ddc58d5fd498dc6ec"
server_key_iv: "6a93ca5a5bbbf7fbf363ee4cff96395c"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.436044 22207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "287e98c805193f6ddc58d5fd498dc6ec"
server_key_iv: "6a93ca5a5bbbf7fbf363ee4cff96395c"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.437268 22207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:04:17.437919 22234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.438062 22207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:17.438144 22207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "287e98c805193f6ddc58d5fd498dc6ec"
server_key_iv: "6a93ca5a5bbbf7fbf363ee4cff96395c"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.438201 22207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:17.452996 22207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:17.454972 22207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:17.455101 22207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:17.455288 22207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:17.455569 22207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:17.455600 22207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.455627 22207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:17.455646 22207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.461014 22207 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:37059
I20260808 17:04:17.461078 22425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:37059 every 8 connection(s)
I20260808 17:04:17.461355 22207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:04:17 dist-test-slave-2j7r krb5kdc[21842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208657, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:17.467398 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 22207
I20260808 17:04:17.467501 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:04:17.467662 31496 external_mini_cluster.cc:1428] Setting key 0254b2e22f331547f672ffd763a7ecc6
I20260808 17:04:17.469894 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.462026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:49207 (local address 127.30.194.62:40309)
0808 17:04:17.462092 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.462095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.462251 (+ 156us) server_negotiation.cc:407] Connection header received
0808 17:04:17.462749 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.462750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.462763 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.462818 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:17.463693 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.464177 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.465148 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.465359 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.465413 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.467305 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:17.467306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:17.467307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:17.467314 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:17.468651 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:17.469009 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:17.469010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:17.469010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:17.469069 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:17.469306 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:17.469307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:17.469307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:17.469426 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:17.469484 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.469730 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.469807 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:04:17.470253 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.461970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:49207)
0808 17:04:17.462136 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:17.462177 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:17.462654 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:17.462886 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:17.462890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:17.463430 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:17.463602 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.463606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.464258 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.464259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.465038 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.465041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.465427 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.465427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.465545 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.465962 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:17.465970 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:17.467220 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:17.468721 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:17.468728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:17.468730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:17.468943 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:17.469148 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:17.469149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:17.469149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:17.469251 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:17.469481 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:17.469488 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:17.469614 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 17:04:17.469700 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 17:04:17.469846 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260808 17:04:17.470738 22426 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40309
I20260808 17:04:17.470824 22426 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:17.470988 22426 heartbeater.cc:511] Master 127.30.194.62:40309 requested a full tablet report, sending...
I20260808 17:04:17.471347 21905 ts_manager.cc:194] Registered new tserver with Master: b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059)
I20260808 17:04:17.471704 21905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:49207
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:04:17.489944 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40309
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:17.568547 22438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:17.568732 22438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:17.568751 22438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:17.569862 22438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:17.569901 22438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:17.569927 22438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:04:17.571354 22438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:17.571578 22438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:17.571780 22438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:17.576381 22447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:17.576416 22448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:17.576434 22451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:17.576648 22438 server_base.cc:1034] running on GCE node
I20260808 17:04:17.576858 22438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:17.577109 22438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:17.578256 22438 hybrid_clock.cc:648] HybridClock initialized: now 1786208657578244 us; error 28 us; skew 500 ppm
Aug 08 17:04:17 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208657, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:17.580564 22438 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:04:17.580948 22438 webserver.cc:460] Webserver started at http://127.30.194.3:37951/ using document root <none> and password file <none>
I20260808 17:04:17.581187 22438 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:17.581236 22438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:17.581362 22438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:17.582296 22438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "2e8e4d77dcf04a6cb22f473bf088b54d"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "ea3efb925323dc8a1cc203725d69446b"
server_key_iv: "2710d71d8c98efd8a414b353c3390f9d"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.582628 22438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "2e8e4d77dcf04a6cb22f473bf088b54d"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "ea3efb925323dc8a1cc203725d69446b"
server_key_iv: "2710d71d8c98efd8a414b353c3390f9d"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.583823 22438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:04:17.584461 22464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.584604 22438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:04:17.584687 22438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "2e8e4d77dcf04a6cb22f473bf088b54d"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "ea3efb925323dc8a1cc203725d69446b"
server_key_iv: "2710d71d8c98efd8a414b353c3390f9d"
server_key_version: "encryptionkey@0"
I20260808 17:04:17.584750 22438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:17.606230 22438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:17.608193 22438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:17.608321 22438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:17.608520 22438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:17.608795 22438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:17.608829 22438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.608856 22438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:17.608891 22438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:17.614478 22438 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:43667
I20260808 17:04:17.614656 22640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:43667 every 8 connection(s)
I20260808 17:04:17.614862 22438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:04:17 dist-test-slave-2j7r krb5kdc[21842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208657, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:17.623395 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.615692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:44697 (local address 127.30.194.62:40309)
0808 17:04:17.615738 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.615740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.615928 (+ 188us) server_negotiation.cc:407] Connection header received
0808 17:04:17.616574 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.616575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.616589 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.616639 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:17.617646 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.618138 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.619081 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.619281 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.619336 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.621185 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:17.621186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:17.621187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:17.621195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:17.622177 (+ 982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:17.622528 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:17.622529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:17.622530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:17.622585 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:17.622794 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:17.622795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:17.622795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:17.622907 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:17.622963 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.623208 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.623290 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:04:17.623606 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.615639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:44697)
0808 17:04:17.615797 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:17.615837 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:17.616474 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:17.616703 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:17.616708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:17.617374 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:17.617554 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.617558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.618219 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.618220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.618999 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.619002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.619350 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.619351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.619477 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.619829 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:17.619836 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:17.621071 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:17.622249 (+ 1178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:17.622256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:17.622258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:17.622461 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:17.622638 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:17.622639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:17.622640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:17.622740 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:17.622962 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:17.622968 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:17.623094 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 17:04:17.623180 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 17:04:17.623321 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260808 17:04:17.624079 22644 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40309
I20260808 17:04:17.624184 22644 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:17.624229 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 22438
I20260808 17:04:17.624339 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:04:17.624368 22644 heartbeater.cc:511] Master 127.30.194.62:40309 requested a full tablet report, sending...
I20260808 17:04:17.624544 31496 external_mini_cluster.cc:1428] Setting key c014d1b87909f6a036e8295877436e41
I20260808 17:04:17.624747 21905 ts_manager.cc:194] Registered new tserver with Master: 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667)
I20260808 17:04:17.625188 21905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:44697
I20260808 17:04:17.625464 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:04:17.631456 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.626689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37278 (local address 127.30.194.62:40309)
0808 17:04:17.626740 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.626741 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.626859 (+ 118us) server_negotiation.cc:407] Connection header received
0808 17:04:17.626900 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.626901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.626910 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.626954 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:17.627424 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.627860 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.628488 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.628653 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.628696 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.629686 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:17.629687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:17.629687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:17.629692 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:17.630667 (+ 975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:17.630941 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:17.630941 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:17.630942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:17.630981 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:17.631097 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:17.631098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:17.631098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:17.631209 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:17.631282 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.631350 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.631402 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:04:17.633052 21904 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:04:17.634578 21904 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:04:17.640244 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.636488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:43667 (local address 127.0.0.1:39476)
0808 17:04:17.636631 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:17.636666 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:17.636729 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:17.637037 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:17.637040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:17.637042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:17.637123 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.637125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.638091 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.638092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.639024 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.639027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.639943 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.639945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.640026 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.640039 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:04:17.640112 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 17:04:17.640158 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":45,"threads_started":1}
I20260808 17:04:17.640249 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.636543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39476 (local address 127.30.194.3:43667)
0808 17:04:17.636878 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.636882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.636891 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:04:17.636924 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.636926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.636940 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.637015 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:17.637198 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.637988 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.639118 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.639840 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.639932 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.639989 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.640140 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.640176 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":89,"threads_started":1}
I20260808 17:04:17.640789 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.636524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:37059 (local address 127.0.0.1:38310)
0808 17:04:17.636840 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:17.636851 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:17.636901 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:17.637299 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:17.637300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:17.637301 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:17.637363 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.637364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.638643 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.638644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.639601 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.639604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.640534 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.640535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.640602 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.640606 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:17.640678 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:04:17.640718 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":26,"thread_start_us":68,"threads_started":1}
I20260808 17:04:17.640887 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.636715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33709 (local address 127.0.0.1:32992)
0808 17:04:17.637155 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:17.637167 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:17.637214 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:17.637468 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:17.637469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:17.637471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:17.637506 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.637507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.638697 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.638698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.639667 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.639670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.640619 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.640620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.640683 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.640688 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:04:17.640742 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:04:17.640786 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":69,"threads_started":1}
I20260808 17:04:17.640914 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.636833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32992 (local address 127.30.194.1:33709)
0808 17:04:17.637035 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.637040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.637267 (+ 227us) server_negotiation.cc:407] Connection header received
0808 17:04:17.637309 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.637311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.637326 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.637393 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:17.637575 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.638593 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.639742 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.640470 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.640529 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.640585 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.640779 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.640832 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260808 17:04:17.641144 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.636662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38310 (local address 127.30.194.2:37059)
0808 17:04:17.637058 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.637062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.637071 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:04:17.637126 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.637128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.637144 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.637222 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:17.637436 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.638532 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.639699 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.640417 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.640489 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.640543 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.640998 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.641044 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":92,"threads_started":1}
I20260808 17:04:17.641312 22540 tablet_service.cc:1467] Processing CreateTablet for tablet 154948a40b3547958954f0f92e8906ca (DEFAULT_TABLE table=test-table [id=521e2d42cbfc4d098da57a5231061f07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:17.641641 22540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 154948a40b3547958954f0f92e8906ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:17.641577 22096 tablet_service.cc:1467] Processing CreateTablet for tablet 154948a40b3547958954f0f92e8906ca (DEFAULT_TABLE table=test-table [id=521e2d42cbfc4d098da57a5231061f07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:17.641904 22096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 154948a40b3547958954f0f92e8906ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:17.641983 22316 tablet_service.cc:1467] Processing CreateTablet for tablet 154948a40b3547958954f0f92e8906ca (DEFAULT_TABLE table=test-table [id=521e2d42cbfc4d098da57a5231061f07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:17.642302 22316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 154948a40b3547958954f0f92e8906ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:17.644706 22664 tablet_bootstrap.cc:492] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: Bootstrap starting.
I20260808 17:04:17.645215 22667 tablet_bootstrap.cc:492] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Bootstrap starting.
I20260808 17:04:17.645444 22664 tablet_bootstrap.cc:654] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:17.645754 22667 tablet_bootstrap.cc:654] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:17.645963 22664 log.cc:824] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:17.646207 22667 log.cc:824] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:17.646531 22666 tablet_bootstrap.cc:492] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: Bootstrap starting.
I20260808 17:04:17.646739 22664 tablet_bootstrap.cc:492] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: No bootstrap required, opened a new log
I20260808 17:04:17.646835 22664 ts_tablet_manager.cc:1397] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:04:17.647110 22666 tablet_bootstrap.cc:654] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:17.647113 22667 tablet_bootstrap.cc:492] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: No bootstrap required, opened a new log
I20260808 17:04:17.647176 22667 ts_tablet_manager.cc:1397] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:04:17.647596 22666 log.cc:824] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:17.648236 22666 tablet_bootstrap.cc:492] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: No bootstrap required, opened a new log
I20260808 17:04:17.648293 22666 ts_tablet_manager.cc:1397] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:04:17.648923 22664 raft_consensus.cc:348] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:17.649132 22664 raft_consensus.cc:374] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:17.649191 22664 raft_consensus.cc:729] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9e52d94e93b4fb08ecb71ddd1e7a439, State: Initialized, Role: FOLLOWER
I20260808 17:04:17.649312 22664 consensus_queue.cc:260] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:17.649364 22667 raft_consensus.cc:348] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:17.649518 22667 raft_consensus.cc:374] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:17.649564 22667 raft_consensus.cc:729] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 289bfa4ac1fc4059b99ac51c06b7c9cd, State: Initialized, Role: FOLLOWER
I20260808 17:04:17.649608 22664 ts_tablet_manager.cc:1428] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 17:04:17.649740 22426 heartbeater.cc:503] Master 127.30.194.62:40309 was elected leader, sending a full tablet report...
I20260808 17:04:17.649735 22667 consensus_queue.cc:260] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:17.649866 22666 raft_consensus.cc:348] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:17.650007 22666 raft_consensus.cc:374] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:17.650012 22198 heartbeater.cc:503] Master 127.30.194.62:40309 was elected leader, sending a full tablet report...
I20260808 17:04:17.650049 22666 raft_consensus.cc:729] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8e4d77dcf04a6cb22f473bf088b54d, State: Initialized, Role: FOLLOWER
I20260808 17:04:17.650154 22666 consensus_queue.cc:260] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:17.650012 22667 ts_tablet_manager.cc:1428] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:04:17.650447 22666 ts_tablet_manager.cc:1428] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:04:17.650575 22644 heartbeater.cc:503] Master 127.30.194.62:40309 was elected leader, sending a full tablet report...
W20260808 17:04:17.711895 22427 tablet.cc:2369] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:04:17.816468 22202 tablet.cc:2369] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:04:17.865469 22645 tablet.cc:2369] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:04:17.915093 22676 raft_consensus.cc:482] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:17.915213 22676 raft_consensus.cc:504] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:17.915599 22676 leader_election.cc:288] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667), 289bfa4ac1fc4059b99ac51c06b7c9cd (127.30.194.1:33709)
I20260808 17:04:17.919238 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.915775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:43667 (local address 127.30.194.2:45723)
0808 17:04:17.915846 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:17.915857 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:17.915907 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:17.916067 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:17.916068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:17.916070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:17.916137 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.916138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.917030 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.917031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.918265 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.918268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.918995 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.918996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.919076 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.919080 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:17.919125 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:04:17.919167 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":6}
I20260808 17:04:17.919247 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.915819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:45723 (local address 127.30.194.3:43667)
0808 17:04:17.915861 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.915863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.915899 (+ 36us) server_negotiation.cc:407] Connection header received
0808 17:04:17.915988 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.915990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.915999 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.916054 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:17.916199 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.916953 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.918353 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.918908 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.918976 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.918984 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.919138 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.919182 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:04:17.919693 22570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "154948a40b3547958954f0f92e8906ca" candidate_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" is_pre_election: true
I20260808 17:04:17.919865 22570 raft_consensus.cc:2454] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9e52d94e93b4fb08ecb71ddd1e7a439 in term 0.
I20260808 17:04:17.920087 22239 leader_election.cc:302] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e8e4d77dcf04a6cb22f473bf088b54d, b9e52d94e93b4fb08ecb71ddd1e7a439; no voters:
I20260808 17:04:17.920212 22676 raft_consensus.cc:2781] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:04:17.920282 22676 raft_consensus.cc:482] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:04:17.920303 22676 raft_consensus.cc:3037] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:17.919303 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.915861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:52927 (local address 127.30.194.1:33709)
0808 17:04:17.915920 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.915922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.916044 (+ 122us) server_negotiation.cc:407] Connection header received
0808 17:04:17.916069 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.916070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.916079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.916124 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:17.916247 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.916957 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.918350 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.918908 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.918977 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.918984 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.919202 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.919244 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:04:17.920576 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.915783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33709 (local address 127.30.194.2:52927)
0808 17:04:17.915956 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:17.915965 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:17.916013 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:17.916136 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:17.916137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:17.916138 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:17.916188 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.916189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.917027 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.917028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.918264 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:17.918268 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.919040 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.919042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:17.919120 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.919124 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:17.920463 (+ 1339us) client_negotiation.cc:240] Negotiation successful
0808 17:04:17.920511 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":71,"thread_start_us":90,"threads_started":1}
I20260808 17:04:17.920882 22129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "154948a40b3547958954f0f92e8906ca" candidate_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" is_pre_election: true
I20260808 17:04:17.921011 22129 raft_consensus.cc:2454] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9e52d94e93b4fb08ecb71ddd1e7a439 in term 0.
I20260808 17:04:17.921159 22676 raft_consensus.cc:504] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:17.921279 22676 leader_election.cc:288] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667), 289bfa4ac1fc4059b99ac51c06b7c9cd (127.30.194.1:33709)
I20260808 17:04:17.921412 22570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "154948a40b3547958954f0f92e8906ca" candidate_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d"
I20260808 17:04:17.921430 22129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "154948a40b3547958954f0f92e8906ca" candidate_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd"
I20260808 17:04:17.921483 22570 raft_consensus.cc:3037] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:17.921504 22129 raft_consensus.cc:3037] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:17.922138 22570 raft_consensus.cc:2454] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9e52d94e93b4fb08ecb71ddd1e7a439 in term 1.
I20260808 17:04:17.922250 22129 raft_consensus.cc:2454] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9e52d94e93b4fb08ecb71ddd1e7a439 in term 1.
I20260808 17:04:17.922293 22239 leader_election.cc:302] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e8e4d77dcf04a6cb22f473bf088b54d, b9e52d94e93b4fb08ecb71ddd1e7a439; no voters:
I20260808 17:04:17.922432 22676 raft_consensus.cc:2781] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:04:17.922549 22676 raft_consensus.cc:686] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 1 LEADER]: Becoming Leader. State: Replica: b9e52d94e93b4fb08ecb71ddd1e7a439, State: Running, Role: LEADER
I20260808 17:04:17.922639 22676 consensus_queue.cc:237] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:17.923393 21904 catalog_manager.cc:5495] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } health_report { overall_health: UNKNOWN } } }
I20260808 17:04:17.992799 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:17.990824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38318 (local address 127.30.194.2:37059)
0808 17:04:17.990881 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:17.990883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:17.990890 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:04:17.990923 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:17.990924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:17.990942 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:17.990996 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:04:17.991093 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.991552 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.992215 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:17.992394 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:17.992436 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:17.992536 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:04:17.992640 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:04:17.992701 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:17.992711 (+ 10us) server_negotiation.cc:299] Negotiation successful
0808 17:04:17.992728 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:04:17.994424 22570 raft_consensus.cc:1261] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d [term 1 FOLLOWER]: Refusing update from remote peer b9e52d94e93b4fb08ecb71ddd1e7a439: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:17.994443 22129 raft_consensus.cc:1261] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Refusing update from remote peer b9e52d94e93b4fb08ecb71ddd1e7a439: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:17.994761 22676 consensus_queue.cc:1035] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:17.994892 22683 consensus_queue.cc:1035] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [LEADER]: Connected to new peer: Peer: permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:17.999284 21904 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:04:17.999367 21904 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:04:18.000063 21904 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Sending DeleteTablet for 3 replicas of tablet 154948a40b3547958954f0f92e8906ca
I20260808 17:04:18.000391 22096 tablet_service.cc:1514] Processing DeleteTablet for tablet 154948a40b3547958954f0f92e8906ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:17 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:32992
I20260808 17:04:18.000379 22540 tablet_service.cc:1514] Processing DeleteTablet for tablet 154948a40b3547958954f0f92e8906ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:17 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:39476
I20260808 17:04:18.000703 22704 tablet_replica.cc:330] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: stopping tablet replica
I20260808 17:04:18.000702 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 22005
I20260808 17:04:18.000703 22316 tablet_service.cc:1514] Processing DeleteTablet for tablet 154948a40b3547958954f0f92e8906ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:17 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:38310
I20260808 17:04:18.000801 22704 raft_consensus.cc:2229] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:18.000902 22704 raft_consensus.cc:2258] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:18.001022 22703 tablet_replica.cc:330] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: stopping tablet replica
I20260808 17:04:18.001113 22703 raft_consensus.cc:2229] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:18.003641 22703 raft_consensus.cc:2258] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:18.009943 22703 ts_tablet_manager.cc:1905] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:18.011215 22707 tablet_replica.cc:330] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: stopping tablet replica
I20260808 17:04:18.011291 22707 raft_consensus.cc:2229] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:04:18.011372 22707 raft_consensus.cc:2258] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:18.011448 22703 ts_tablet_manager.cc:1918] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:18.011499 22703 log.cc:1196] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/154948a40b3547958954f0f92e8906ca
I20260808 17:04:18.011725 22703 ts_tablet_manager.cc:1939] T 154948a40b3547958954f0f92e8906ca P 2e8e4d77dcf04a6cb22f473bf088b54d: Deleting consensus metadata
I20260808 17:04:18.012079 21882 catalog_manager.cc:4841] TS 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667): tablet 154948a40b3547958954f0f92e8906ca (table test-table [id=521e2d42cbfc4d098da57a5231061f07]) successfully deleted
I20260808 17:04:18.012750 22707 ts_tablet_manager.cc:1905] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:18.014055 22707 ts_tablet_manager.cc:1918] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:18.014117 22707 log.cc:1196] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/154948a40b3547958954f0f92e8906ca
I20260808 17:04:18.014436 22707 ts_tablet_manager.cc:1939] T 154948a40b3547958954f0f92e8906ca P b9e52d94e93b4fb08ecb71ddd1e7a439: Deleting consensus metadata
I20260808 17:04:18.015385 21879 catalog_manager.cc:4841] TS b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059): tablet 154948a40b3547958954f0f92e8906ca (table test-table [id=521e2d42cbfc4d098da57a5231061f07]) successfully deleted
W20260808 17:04:18.033454 21879 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:33709 (error 108)
I20260808 17:04:18.033643 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 22207
I20260808 17:04:18.034194 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.033793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33709 (local address 127.0.0.1:32996)
0808 17:04:18.034107 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:18.034157 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:33709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291}
W20260808 17:04:18.036952 21879 catalog_manager.cc:4584] TS 289bfa4ac1fc4059b99ac51c06b7c9cd (127.30.194.1:33709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 154948a40b3547958954f0f92e8906ca: Network error: Client connection negotiation failed: client connection to 127.30.194.1:33709: connect: Connection refused (error 111)
I20260808 17:04:18.048722 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 22438
I20260808 17:04:18.057899 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 21859
I20260808 17:04:18.068383 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40309
--webserver_interface=127.30.194.62
--webserver_port=39431
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:40309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:18.147387 22710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:18.147567 22710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:18.147588 22710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:18.148699 22710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:18.148738 22710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:18.148751 22710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:18.148774 22710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:18.148797 22710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:18.150213 22710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:40309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=39431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:18.150436 22710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:18.150636 22710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:18.158953 22718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:18.158953 22719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:18.159050 22723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:18.159206 22710 server_base.cc:1034] running on GCE node
I20260808 17:04:18.159453 22710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:18.159720 22710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:18.160856 22710 hybrid_clock.cc:648] HybridClock initialized: now 1786208658160844 us; error 27 us; skew 500 ppm
Aug 08 17:04:18 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208658, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:18.167099 22710 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:04:18.167511 22710 webserver.cc:460] Webserver started at http://127.30.194.62:39431/ using document root <none> and password file <none>
I20260808 17:04:18.167690 22710 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:18.167738 22710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:18.168949 22710 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:04:18.169575 22734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.169714 22710 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.169795 22710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "c7b0929cd4a74a77baf22d08a08b510b"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "5fbc5d291cbc82efb59e75abd22c783f"
server_key_iv: "5530db40daa64dfb490c08a9bb9b4fe1"
server_key_version: "encryptionkey@0"
I20260808 17:04:18.170150 22710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:18.190349 22710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:18.192301 22710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:18.192428 22710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:18.196213 22710 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:40309
I20260808 17:04:18.196254 22813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:40309 every 8 connection(s)
I20260808 17:04:18.196578 22710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:04:18.198166 22814 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:04:18.198768 22814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Bootstrap starting.
I20260808 17:04:18.203049 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 22710
I20260808 17:04:18.203287 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:33709
--local_ip_for_outbound_sockets=127.30.194.1
--tserver_master_addrs=127.30.194.62:40309
--webserver_port=34321
--webserver_interface=127.30.194.1
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:18.206759 22814 log.cc:824] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:18.208513 22814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:04:18.208763 22814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Bootstrap complete.
I20260808 17:04:18.210825 22814 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:18.211051 22814 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b0929cd4a74a77baf22d08a08b510b, State: Initialized, Role: FOLLOWER
I20260808 17:04:18.211194 22814 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:18.211266 22814 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:04:18.211320 22814 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:04:18.211369 22814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:04:18.212334 22814 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:18.212432 22814 leader_election.cc:302] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7b0929cd4a74a77baf22d08a08b510b; no voters:
I20260808 17:04:18.212582 22814 leader_election.cc:288] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 17:04:18.212641 22823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:04:18.212774 22823 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 2 LEADER]: Becoming Leader. State: Replica: c7b0929cd4a74a77baf22d08a08b510b, State: Running, Role: LEADER
I20260808 17:04:18.212819 22814 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:04:18.212882 22823 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:18.213158 22825 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c7b0929cd4a74a77baf22d08a08b510b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } } }
I20260808 17:04:18.213244 22825 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:18.213155 22823 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7b0929cd4a74a77baf22d08a08b510b. Latest consensus state: current_term: 2 leader_uuid: "c7b0929cd4a74a77baf22d08a08b510b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } } }
I20260808 17:04:18.213413 22823 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:18.213507 22834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:04:18.214370 22834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:04:18.214468 22834 catalog_manager.cc:1259] Loaded cluster ID: 3063efba345d4478872cf3c91a939205
I20260808 17:04:18.214509 22834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:04:18.215014 22834 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 17:04:18.219620 22834 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 17:04:18.221962 22834 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:04:18.222159 22834 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Loaded TSK: 0
I20260808 17:04:18.222401 22834 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:04:18.284397 22818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:18.284578 22818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:18.284608 22818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:18.285729 22818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:18.285769 22818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:18.285804 22818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:04:18.287226 22818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:33709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=34321
--tserver_master_addrs=127.30.194.62:40309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:18.287447 22818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:18.287642 22818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:18.292194 22869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:18.292246 22874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:18.292203 22870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:18.292752 22818 server_base.cc:1034] running on GCE node
I20260808 17:04:18.292922 22818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:18.293175 22818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:18.294320 22818 hybrid_clock.cc:648] HybridClock initialized: now 1786208658294307 us; error 31 us; skew 500 ppm
Aug 08 17:04:18 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208658, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:18.296754 22818 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:04:18.297154 22818 webserver.cc:460] Webserver started at http://127.30.194.1:34321/ using document root <none> and password file <none>
I20260808 17:04:18.297385 22818 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:18.297446 22818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:18.298771 22818 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:04:18.299422 22889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.299577 22818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:04:18.299660 22818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "b7b2bfed18ce1a027a0991ed423f6c77"
server_key_iv: "e991e3390bc9e7a37826baac055c4444"
server_key_version: "encryptionkey@0"
I20260808 17:04:18.299932 22818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:18.320916 22818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:18.322716 22818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:18.322846 22818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:18.323032 22818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:18.323417 22903 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:04:18.325207 22818 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:04:18.325249 22818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 17:04:18.325278 22818 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:04:18.325883 22818 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:04:18.325924 22818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 17:04:18.326036 22903 tablet_bootstrap.cc:492] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Bootstrap starting.
I20260808 17:04:18.332414 22818 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:33709
I20260808 17:04:18.332528 23083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:33709 every 8 connection(s)
I20260808 17:04:18.332813 22818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:04:18.336447 22903 log.cc:824] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:18.338519 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 22818
I20260808 17:04:18.338788 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:37059
--local_ip_for_outbound_sockets=127.30.194.2
--tserver_master_addrs=127.30.194.62:40309
--webserver_port=33147
--webserver_interface=127.30.194.2
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:18.338835 22903 tablet_bootstrap.cc:492] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:04:18.339160 22903 tablet_bootstrap.cc:492] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Bootstrap complete.
Aug 08 17:04:18 dist-test-slave-2j7r krb5kdc[21842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208658, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:18.340129 22903 ts_tablet_manager.cc:1397] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.006s
I20260808 17:04:18.341324 22903 raft_consensus.cc:348] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.341638 22903 raft_consensus.cc:729] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 289bfa4ac1fc4059b99ac51c06b7c9cd, State: Initialized, Role: FOLLOWER
I20260808 17:04:18.341781 22903 consensus_queue.cc:260] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.342038 22903 ts_tablet_manager.cc:1428] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260808 17:04:18.343896 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.333514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:39583 (local address 127.30.194.62:40309)
0808 17:04:18.334285 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:18.334292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:18.334324 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:04:18.334421 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:18.334426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:18.334464 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:18.334574 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:18.335190 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.336031 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.337305 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.337685 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.337777 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.340714 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:18.340719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:18.340719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:18.340730 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:18.342359 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:18.342719 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:18.342722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:18.342723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:18.342816 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:18.343034 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:18.343035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:18.343036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:18.343250 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:18.343331 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:18.343599 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 17:04:18.343720 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":71,"threads_started":1}
I20260808 17:04:18.343896 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.333443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:39583)
0808 17:04:18.333690 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:18.333720 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:18.334136 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:18.334618 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:18.334623 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:18.334894 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:18.335072 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.335076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.336140 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.336141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.337228 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.337231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.337779 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.337780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.337922 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.338382 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:18.338393 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:18.340615 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:18.342437 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:18.342444 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:18.342446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:18.342647 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:18.342874 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:18.342875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:18.342876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:18.342979 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:18.343313 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:18.343320 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:18.343448 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 17:04:18.343534 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 17:04:18.343697 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":96,"threads_started":1}
I20260808 17:04:18.344542 23085 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40309
I20260808 17:04:18.344624 23085 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:18.344761 23085 heartbeater.cc:511] Master 127.30.194.62:40309 requested a full tablet report, sending...
I20260808 17:04:18.345536 22759 ts_manager.cc:194] Registered new tserver with Master: 289bfa4ac1fc4059b99ac51c06b7c9cd (127.30.194.1:33709)
I20260808 17:04:18.346726 22759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:39583
I20260808 17:04:18.347615 23085 heartbeater.cc:503] Master 127.30.194.62:40309 was elected leader, sending a full tablet report...
Aug 08 17:04:18 dist-test-slave-2j7r krb5kdc[21842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208658, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
I20260808 17:04:18.354377 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.347042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.30.194.1:33709)
0808 17:04:18.347180 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:18.347185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:18.347193 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:04:18.347223 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:18.347224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:18.347239 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:18.347322 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:18.347950 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.348600 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.349737 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.350008 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.350058 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.351956 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:18.351963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:18.351963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:18.351969 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:18.353291 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:18.353635 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:18.353638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:18.353638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:18.353689 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:18.353861 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:18.353861 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:18.353862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:18.353973 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:18.354043 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:18.354202 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 17:04:18.354277 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260808 17:04:18.354374 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.346159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33709 (local address 127.0.0.1:33010)
0808 17:04:18.346297 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:18.346329 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:18.346389 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:18.347338 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:18.347346 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:18.347666 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:18.347723 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.347726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.348681 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.348682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.349652 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.349655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.350081 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.350082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.350161 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.350527 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:18.350532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:18.351877 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:18.353376 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:18.353383 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:18.353385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:18.353565 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:18.353744 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:18.353745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:18.353745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:18.353805 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:18.354032 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:18.354039 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:18.354090 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 17:04:18.354162 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:04:18.354278 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260808 17:04:18.354859 22981 tablet_service.cc:1514] Processing DeleteTablet for tablet 154948a40b3547958954f0f92e8906ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:17 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:33010
I20260808 17:04:18.355274 23106 tablet_replica.cc:330] stopping tablet replica
I20260808 17:04:18.355381 23106 raft_consensus.cc:2229] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:18.355450 23106 raft_consensus.cc:2258] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:18.355747 23106 ts_tablet_manager.cc:1905] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:18.357788 23106 ts_tablet_manager.cc:1918] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:18.357860 23106 log.cc:1196] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/154948a40b3547958954f0f92e8906ca
I20260808 17:04:18.358107 23106 ts_tablet_manager.cc:1939] T 154948a40b3547958954f0f92e8906ca P 289bfa4ac1fc4059b99ac51c06b7c9cd: Deleting consensus metadata
I20260808 17:04:18.358580 22735 catalog_manager.cc:4841] TS 289bfa4ac1fc4059b99ac51c06b7c9cd (127.30.194.1:33709): tablet 154948a40b3547958954f0f92e8906ca (table test-table [id=521e2d42cbfc4d098da57a5231061f07]) successfully deleted
W20260808 17:04:18.421887 23099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:18.422066 23099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:18.422087 23099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:18.423189 23099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:18.423228 23099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:18.423254 23099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:04:18.424706 23099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:37059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=33147
--tserver_master_addrs=127.30.194.62:40309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:18.424921 23099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:18.425129 23099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:18.429584 23111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:18.429625 23117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:18.429602 23114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:18.429634 23099 server_base.cc:1034] running on GCE node
I20260808 17:04:18.429939 23099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:18.430161 23099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:18.431288 23099 hybrid_clock.cc:648] HybridClock initialized: now 1786208658431277 us; error 28 us; skew 500 ppm
Aug 08 17:04:18 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208658, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:18.433465 23099 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:04:18.433835 23099 webserver.cc:460] Webserver started at http://127.30.194.2:33147/ using document root <none> and password file <none>
I20260808 17:04:18.434043 23099 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:18.434098 23099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:18.435285 23099 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:18.435761 23127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.435918 23099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.436007 23099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "287e98c805193f6ddc58d5fd498dc6ec"
server_key_iv: "6a93ca5a5bbbf7fbf363ee4cff96395c"
server_key_version: "encryptionkey@0"
I20260808 17:04:18.436280 23099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:18.448134 23099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:18.449671 23099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:18.449790 23099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:18.449972 23099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:18.450222 23099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:18.450253 23099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.450278 23099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:18.450302 23099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.455619 23099 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:37059
I20260808 17:04:18.455672 23293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:37059 every 8 connection(s)
I20260808 17:04:18.455956 23099 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:04:18 dist-test-slave-2j7r krb5kdc[21842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208658, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:18.464295 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.456549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:52441 (local address 127.30.194.62:40309)
0808 17:04:18.456600 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:18.456602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:18.456789 (+ 187us) server_negotiation.cc:407] Connection header received
0808 17:04:18.457284 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:18.457286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:18.457303 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:18.457352 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:18.458197 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.458686 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.459948 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.460168 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.460224 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.462039 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:18.462041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:18.462041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:18.462049 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:18.463079 (+ 1030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:18.463433 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:18.463435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:18.463435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:18.463492 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:18.463700 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:18.463701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:18.463701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:18.463811 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:18.463866 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:18.464111 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 17:04:18.464204 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:04:18.464421 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.456540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:52441)
0808 17:04:18.456684 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:18.456715 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:18.457192 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:18.457422 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:18.457426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:18.457940 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:18.458105 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.458109 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.458767 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.458768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.459860 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.459863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.460232 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.460234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.460359 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.460707 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:18.460714 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:18.461952 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:18.463151 (+ 1199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:18.463158 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:18.463160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:18.463357 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:18.463545 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:18.463546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:18.463546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:18.463644 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:18.463864 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:18.463870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:18.463997 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 17:04:18.464081 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 17:04:18.464217 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260808 17:04:18.464676 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 23099
I20260808 17:04:18.464855 23297 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40309
I20260808 17:04:18.464939 23297 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:18.464927 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:43667
--local_ip_for_outbound_sockets=127.30.194.3
--tserver_master_addrs=127.30.194.62:40309
--webserver_port=37951
--webserver_interface=127.30.194.3
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:18.465067 23297 heartbeater.cc:511] Master 127.30.194.62:40309 requested a full tablet report, sending...
I20260808 17:04:18.465440 22759 ts_manager.cc:194] Registered new tserver with Master: b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059)
I20260808 17:04:18.465790 22759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:52441
W20260808 17:04:18.544485 23309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:18.544658 23309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:18.544679 23309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:18.545846 23309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:18.545886 23309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:18.545912 23309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:04:18.547276 23309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:43667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=37951
--tserver_master_addrs=127.30.194.62:40309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:18.547437 23309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:18.547598 23309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:18.552067 23321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:18.552062 23315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:18.552284 23309 server_base.cc:1034] running on GCE node
W20260808 17:04:18.552353 23317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:18.552527 23309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:18.552728 23309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:18.553876 23309 hybrid_clock.cc:648] HybridClock initialized: now 1786208658553862 us; error 28 us; skew 500 ppm
Aug 08 17:04:18 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208658, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:18.556147 23309 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:04:18.556536 23309 webserver.cc:460] Webserver started at http://127.30.194.3:37951/ using document root <none> and password file <none>
I20260808 17:04:18.556697 23309 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:18.556746 23309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:18.557957 23309 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:18.558444 23330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.558599 23309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.558671 23309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "2e8e4d77dcf04a6cb22f473bf088b54d"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "ea3efb925323dc8a1cc203725d69446b"
server_key_iv: "2710d71d8c98efd8a414b353c3390f9d"
server_key_version: "encryptionkey@0"
I20260808 17:04:18.558965 23309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:18.565392 23309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:18.566927 23309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:18.567042 23309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:18.567235 23309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:18.567492 23309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:18.567523 23309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.567540 23309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:18.567562 23309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:18.573122 23309 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:43667
I20260808 17:04:18.573176 23520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:43667 every 8 connection(s)
I20260808 17:04:18.573467 23309 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:04:18 dist-test-slave-2j7r krb5kdc[21842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208658, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:18.579982 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 23309
I20260808 17:04:18.581467 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.574093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:36209 (local address 127.30.194.62:40309)
0808 17:04:18.574165 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:18.574168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:18.574325 (+ 157us) server_negotiation.cc:407] Connection header received
0808 17:04:18.574729 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:18.574730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:18.574740 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:18.574787 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:18.575577 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.576096 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.576987 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.577205 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.577250 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.579022 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:18.579023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:18.579023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:18.579030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:18.580177 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:18.580581 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:18.580582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:18.580583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:18.580637 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:18.580850 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:18.580851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:18.580851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:18.580966 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:18.581022 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:18.581297 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 17:04:18.581381 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:04:18.581610 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.574077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:36209)
0808 17:04:18.574224 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:18.574254 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:18.574656 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:18.574840 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:18.574843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:18.575347 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:18.575512 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.575517 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.576167 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.576167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.576926 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.576929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.577254 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.577255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.577349 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.577691 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:18.577699 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:18.578934 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:18.580248 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:18.580254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:18.580256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:18.580512 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:18.580693 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:18.580694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:18.580694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:18.580795 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:18.581024 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:18.581030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:18.581182 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 17:04:18.581266 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 17:04:18.581417 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260808 17:04:18.582022 23521 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40309
I20260808 17:04:18.582121 23521 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:18.582286 23521 heartbeater.cc:511] Master 127.30.194.62:40309 requested a full tablet report, sending...
I20260808 17:04:18.582644 22759 ts_manager.cc:194] Registered new tserver with Master: 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667)
I20260808 17:04:18.583040 22759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:36209
I20260808 17:04:18.585834 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.580316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37292 (local address 127.30.194.62:40309)
0808 17:04:18.580424 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:18.580426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:18.580461 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:04:18.580524 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:18.580524 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:18.580536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:18.580587 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:18.581080 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.581728 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.582661 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.582904 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.582952 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.583985 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:18.583986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:18.583987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:18.583992 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:18.585029 (+ 1037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:18.585336 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:18.585337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:18.585337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:18.585386 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:18.585500 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:18.585500 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:18.585500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:18.585580 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:18.585683 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:18.585707 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 17:04:18.585765 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":65,"threads_started":1}
I20260808 17:04:18.586475 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:04:18.588604 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.586749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.30.194.62:40309)
0808 17:04:18.586791 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:18.586793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:18.586882 (+ 89us) server_negotiation.cc:407] Connection header received
0808 17:04:18.586920 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:18.586921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:18.586945 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:18.586997 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:04:18.587064 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.587476 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.588085 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.588249 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.588287 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.588404 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:04:18.588482 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:04:18.588532 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:18.588546 (+ 14us) server_negotiation.cc:299] Negotiation successful
0808 17:04:18.588561 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:04:18.588835 22759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:04:18.590377 22759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:04:18.592509 22981 tablet_service.cc:1467] Processing CreateTablet for tablet 460b0365ddfa4ecebf2f33d884b9e63c (DEFAULT_TABLE table=test-table [id=5a200f7c06fb4c5a8e571de91fede343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:18.592739 22981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460b0365ddfa4ecebf2f33d884b9e63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:18.594307 22903 tablet_bootstrap.cc:492] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Bootstrap starting.
I20260808 17:04:18.594650 22903 tablet_bootstrap.cc:654] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:18.595475 22903 tablet_bootstrap.cc:492] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: No bootstrap required, opened a new log
I20260808 17:04:18.595538 22903 ts_tablet_manager.cc:1397] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:18.595741 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.592016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:43667 (local address 127.0.0.1:39482)
0808 17:04:18.592050 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:18.592061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:18.592101 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:18.592533 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:18.592534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:18.592535 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:18.592571 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.592572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.593606 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.593607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.594630 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.594633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.595509 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.595510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.595571 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.595592 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 17:04:18.595633 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 17:04:18.595677 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 17:04:18.595763 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.592081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39482 (local address 127.30.194.3:43667)
0808 17:04:18.592397 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:18.592401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:18.592409 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:04:18.592440 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:18.592441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:18.592452 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:18.592516 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:18.592693 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.593525 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.594793 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.595433 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.595494 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.595646 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:18.595670 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 17:04:18.595703 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":71,"threads_started":1}
I20260808 17:04:18.595885 22903 raft_consensus.cc:348] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.595976 22903 raft_consensus.cc:374] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:18.596031 22903 raft_consensus.cc:729] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 289bfa4ac1fc4059b99ac51c06b7c9cd, State: Initialized, Role: FOLLOWER
I20260808 17:04:18.596091 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.592103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:37059 (local address 127.0.0.1:38330)
0808 17:04:18.592244 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:18.592255 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:18.592305 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:18.592619 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:18.592620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:18.592622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:18.592671 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.592672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.593921 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.593921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.595014 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.595017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.595881 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.595883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.595946 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.595950 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:18.595984 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 17:04:18.596025 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260808 17:04:18.596081 22903 consensus_queue.cc:260] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.596197 22903 ts_tablet_manager.cc:1428] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:18.596413 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.592195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.30.194.2:37059)
0808 17:04:18.592380 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:18.592385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:18.592397 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:04:18.592437 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:18.592439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:18.592456 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:18.592537 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:18.592781 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.593808 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.595093 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.595762 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.596194 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.596264 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:18.596293 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 17:04:18.596333 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260808 17:04:18.596544 23407 tablet_service.cc:1467] Processing CreateTablet for tablet 460b0365ddfa4ecebf2f33d884b9e63c (DEFAULT_TABLE table=test-table [id=5a200f7c06fb4c5a8e571de91fede343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:18.596850 23407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460b0365ddfa4ecebf2f33d884b9e63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:18.597069 23203 tablet_service.cc:1467] Processing CreateTablet for tablet 460b0365ddfa4ecebf2f33d884b9e63c (DEFAULT_TABLE table=test-table [id=5a200f7c06fb4c5a8e571de91fede343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:18.597394 23203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460b0365ddfa4ecebf2f33d884b9e63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:18.600129 23541 tablet_bootstrap.cc:492] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: Bootstrap starting.
I20260808 17:04:18.600806 23541 tablet_bootstrap.cc:654] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:18.600783 23543 tablet_bootstrap.cc:492] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: Bootstrap starting.
I20260808 17:04:18.601301 23541 log.cc:824] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:18.601351 23543 tablet_bootstrap.cc:654] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:18.601856 23543 log.cc:824] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:18.601998 23541 tablet_bootstrap.cc:492] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: No bootstrap required, opened a new log
I20260808 17:04:18.602080 23541 ts_tablet_manager.cc:1397] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:04:18.602527 23543 tablet_bootstrap.cc:492] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: No bootstrap required, opened a new log
I20260808 17:04:18.602583 23543 ts_tablet_manager.cc:1397] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:04:18.603685 23541 raft_consensus.cc:348] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.603818 23541 raft_consensus.cc:374] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:18.603854 23541 raft_consensus.cc:729] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8e4d77dcf04a6cb22f473bf088b54d, State: Initialized, Role: FOLLOWER
I20260808 17:04:18.603967 23541 consensus_queue.cc:260] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.604039 23543 raft_consensus.cc:348] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.604190 23543 raft_consensus.cc:374] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:18.604234 23543 raft_consensus.cc:729] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9e52d94e93b4fb08ecb71ddd1e7a439, State: Initialized, Role: FOLLOWER
I20260808 17:04:18.604244 23521 heartbeater.cc:503] Master 127.30.194.62:40309 was elected leader, sending a full tablet report...
I20260808 17:04:18.604336 23543 consensus_queue.cc:260] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.604547 23541 ts_tablet_manager.cc:1428] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:04:18.604583 23297 heartbeater.cc:503] Master 127.30.194.62:40309 was elected leader, sending a full tablet report...
I20260808 17:04:18.604878 23543 ts_tablet_manager.cc:1428] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260808 17:04:18.706454 23304 tablet.cc:2369] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:04:18.823972 23523 tablet.cc:2369] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:04:18.833510 23087 tablet.cc:2369] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:04:18.957343 23100 raft_consensus.cc:482] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:18.957491 23100 raft_consensus.cc:504] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.957828 23100 leader_election.cc:288] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667), b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059)
I20260808 17:04:18.960533 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.958086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:57263 (local address 127.30.194.3:43667)
0808 17:04:18.958128 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:18.958130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:18.958195 (+ 65us) server_negotiation.cc:407] Connection header received
0808 17:04:18.958271 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:18.958272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:18.958281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:18.958322 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:18.958447 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.958917 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.959841 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.960211 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.960273 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.960282 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:18.960430 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 17:04:18.960469 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:04:18.960541 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.958057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:38703 (local address 127.30.194.2:37059)
0808 17:04:18.958103 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:18.958105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:18.958172 (+ 67us) server_negotiation.cc:407] Connection header received
0808 17:04:18.958272 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:18.958272 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:18.958280 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:18.958319 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:18.958461 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.958922 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.959864 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.960213 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.960274 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.960282 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:18.960437 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 17:04:18.960474 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:04:18.960577 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.957990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:43667 (local address 127.30.194.1:57263)
0808 17:04:18.958124 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:18.958133 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:18.958201 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:18.958331 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:18.958332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:18.958335 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:18.958392 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.958394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.958990 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.958991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.959755 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.959756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.960285 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.960286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.960351 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.960368 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 17:04:18.960417 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 17:04:18.960459 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":20,"thread_start_us":53,"threads_started":1}
I20260808 17:04:18.960579 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:18.957991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:37059 (local address 127.30.194.1:38703)
0808 17:04:18.958112 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:18.958121 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:18.958192 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:18.958338 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:18.958339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:18.958341 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:18.958408 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.958410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.958992 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.958993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.959781 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:18.959784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:18.960281 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:18.960283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:18.960361 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:18.960379 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 17:04:18.960425 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 17:04:18.960471 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":39,"thread_start_us":58,"threads_started":1}
I20260808 17:04:18.960968 23441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460b0365ddfa4ecebf2f33d884b9e63c" candidate_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" is_pre_election: true
I20260808 17:04:18.960994 23233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460b0365ddfa4ecebf2f33d884b9e63c" candidate_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" is_pre_election: true
I20260808 17:04:18.961155 23441 raft_consensus.cc:2454] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 289bfa4ac1fc4059b99ac51c06b7c9cd in term 0.
I20260808 17:04:18.961164 23233 raft_consensus.cc:2454] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 289bfa4ac1fc4059b99ac51c06b7c9cd in term 0.
I20260808 17:04:18.961366 22891 leader_election.cc:302] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 289bfa4ac1fc4059b99ac51c06b7c9cd, b9e52d94e93b4fb08ecb71ddd1e7a439; no voters:
I20260808 17:04:18.961467 23100 raft_consensus.cc:2781] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:04:18.961529 23100 raft_consensus.cc:482] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:04:18.961550 23100 raft_consensus.cc:3037] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:18.962158 23100 raft_consensus.cc:504] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.962265 23100 leader_election.cc:288] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 1 election: Requested vote from peers 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667), b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059)
I20260808 17:04:18.962420 23233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460b0365ddfa4ecebf2f33d884b9e63c" candidate_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439"
I20260808 17:04:18.962483 23233 raft_consensus.cc:3037] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:18.962477 23441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460b0365ddfa4ecebf2f33d884b9e63c" candidate_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d"
I20260808 17:04:18.962553 23441 raft_consensus.cc:3037] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:18.963189 23233 raft_consensus.cc:2454] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 289bfa4ac1fc4059b99ac51c06b7c9cd in term 1.
I20260808 17:04:18.963228 23441 raft_consensus.cc:2454] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 289bfa4ac1fc4059b99ac51c06b7c9cd in term 1.
I20260808 17:04:18.963372 22891 leader_election.cc:302] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 289bfa4ac1fc4059b99ac51c06b7c9cd, b9e52d94e93b4fb08ecb71ddd1e7a439; no voters:
I20260808 17:04:18.963475 23100 raft_consensus.cc:2781] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:04:18.963572 23100 raft_consensus.cc:686] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 LEADER]: Becoming Leader. State: Replica: 289bfa4ac1fc4059b99ac51c06b7c9cd, State: Running, Role: LEADER
I20260808 17:04:18.963650 23100 consensus_queue.cc:237] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:18.964327 22759 catalog_manager.cc:5495] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 289bfa4ac1fc4059b99ac51c06b7c9cd (127.30.194.1). New cstate: current_term: 1 leader_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } health_report { overall_health: HEALTHY } } }
I20260808 17:04:19.037727 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:19.035636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.30.194.1:33709)
0808 17:04:19.035770 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:19.035771 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:19.035778 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:04:19.035810 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:19.035811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:19.035827 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:19.035888 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:04:19.036017 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:19.036478 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:19.037137 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:19.037327 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:19.037374 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:19.037483 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:04:19.037579 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:04:19.037624 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:19.037652 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 17:04:19.037668 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260808 17:04:19.039183 23233 raft_consensus.cc:1261] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 1 FOLLOWER]: Refusing update from remote peer 289bfa4ac1fc4059b99ac51c06b7c9cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:19.039184 23441 raft_consensus.cc:1261] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d [term 1 FOLLOWER]: Refusing update from remote peer 289bfa4ac1fc4059b99ac51c06b7c9cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:19.039502 23100 consensus_queue.cc:1035] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:19.039660 23557 consensus_queue.cc:1035] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:19.041726 23565 mvcc.cc:205] Tried to move back new op lower bound from 7316310667422310400 to 7316310667115655168. Current Snapshot: MvccSnapshot[applied={T|T < 7316310667422310400}]
I20260808 17:04:19.042670 23567 mvcc.cc:205] Tried to move back new op lower bound from 7316310667422310400 to 7316310667115655168. Current Snapshot: MvccSnapshot[applied={T|T < 7316310667422310400 or (T in {7316310667422310400})}]
I20260808 17:04:19.044220 22759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:37298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:04:19.044296 22759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:37298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:04:19.044965 22759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Sending DeleteTablet for 3 replicas of tablet 460b0365ddfa4ecebf2f33d884b9e63c
I20260808 17:04:19.045280 23407 tablet_service.cc:1514] Processing DeleteTablet for tablet 460b0365ddfa4ecebf2f33d884b9e63c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:19 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:39482
I20260808 17:04:19.045342 22981 tablet_service.cc:1514] Processing DeleteTablet for tablet 460b0365ddfa4ecebf2f33d884b9e63c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:19 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:33010
I20260808 17:04:19.045363 23203 tablet_service.cc:1514] Processing DeleteTablet for tablet 460b0365ddfa4ecebf2f33d884b9e63c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:19 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:38330
I20260808 17:04:19.045537 23580 tablet_replica.cc:330] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: stopping tablet replica
I20260808 17:04:19.045560 23582 tablet_replica.cc:330] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: stopping tablet replica
I20260808 17:04:19.045625 23580 raft_consensus.cc:2229] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:19.045645 23582 raft_consensus.cc:2229] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:19.045652 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 22818
I20260808 17:04:19.045718 23580 raft_consensus.cc:2258] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:19.045732 23582 raft_consensus.cc:2258] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:19.046202 23580 ts_tablet_manager.cc:1905] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:19.047959 23580 ts_tablet_manager.cc:1918] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:19.048030 23580 log.cc:1196] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/460b0365ddfa4ecebf2f33d884b9e63c
I20260808 17:04:19.048352 23580 ts_tablet_manager.cc:1939] T 460b0365ddfa4ecebf2f33d884b9e63c P 2e8e4d77dcf04a6cb22f473bf088b54d: Deleting consensus metadata
I20260808 17:04:19.048784 22739 catalog_manager.cc:4841] TS 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667): tablet 460b0365ddfa4ecebf2f33d884b9e63c (table test-table [id=5a200f7c06fb4c5a8e571de91fede343]) successfully deleted
I20260808 17:04:19.062531 23582 ts_tablet_manager.cc:1905] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:19.063812 23582 ts_tablet_manager.cc:1918] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:19.063879 23582 log.cc:1196] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/460b0365ddfa4ecebf2f33d884b9e63c
I20260808 17:04:19.064082 23582 ts_tablet_manager.cc:1939] T 460b0365ddfa4ecebf2f33d884b9e63c P b9e52d94e93b4fb08ecb71ddd1e7a439: Deleting consensus metadata
I20260808 17:04:19.064484 22735 catalog_manager.cc:4841] TS b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059): tablet 460b0365ddfa4ecebf2f33d884b9e63c (table test-table [id=5a200f7c06fb4c5a8e571de91fede343]) successfully deleted
W20260808 17:04:19.066592 22735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:33709 (error 108)
I20260808 17:04:19.066963 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 23099
I20260808 17:04:19.067042 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:19.066927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:33709 (local address 127.0.0.1:33028)
0808 17:04:19.066973 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:19.067007 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:33709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 17:04:19.068480 22735 catalog_manager.cc:4584] TS 289bfa4ac1fc4059b99ac51c06b7c9cd (127.30.194.1:33709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 460b0365ddfa4ecebf2f33d884b9e63c: Network error: Client connection negotiation failed: client connection to 127.30.194.1:33709: connect: Connection refused (error 111)
I20260808 17:04:19.075250 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 23309
I20260808 17:04:19.084647 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 22710
I20260808 17:04:19.094779 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40309
--webserver_interface=127.30.194.62
--webserver_port=39431
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:40309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:19.173440 23585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:19.173624 23585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:19.173655 23585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:19.174742 23585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:19.174787 23585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:19.174806 23585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:19.174822 23585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:19.174839 23585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:19.176273 23585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:40309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=39431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:19.176504 23585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:19.176689 23585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:19.181114 23599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:19.181123 23592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:19.181128 23593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:19.181211 23585 server_base.cc:1034] running on GCE node
I20260808 17:04:19.181517 23585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:19.181782 23585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:19.182921 23585 hybrid_clock.cc:648] HybridClock initialized: now 1786208659182905 us; error 29 us; skew 500 ppm
Aug 08 17:04:19 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208659, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:19.185263 23585 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:04:19.185644 23585 webserver.cc:460] Webserver started at http://127.30.194.62:39431/ using document root <none> and password file <none>
I20260808 17:04:19.185837 23585 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:19.185884 23585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:19.187032 23585 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:19.187583 23611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:19.187724 23585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:19.187803 23585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "c7b0929cd4a74a77baf22d08a08b510b"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "5fbc5d291cbc82efb59e75abd22c783f"
server_key_iv: "5530db40daa64dfb490c08a9bb9b4fe1"
server_key_version: "encryptionkey@0"
I20260808 17:04:19.188076 23585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:19.217232 23585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:19.219375 23585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:19.219525 23585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:19.223253 23585 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:40309
I20260808 17:04:19.223297 23699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:40309 every 8 connection(s)
I20260808 17:04:19.223644 23585 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:04:19.225200 23703 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:04:19.225823 23703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Bootstrap starting.
I20260808 17:04:19.229532 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 23585
I20260808 17:04:19.229791 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:33709
--local_ip_for_outbound_sockets=127.30.194.1
--tserver_master_addrs=127.30.194.62:40309
--webserver_port=34321
--webserver_interface=127.30.194.1
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:19.234193 23703 log.cc:824] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:19.236238 23703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:04:19.236513 23703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b: Bootstrap complete.
I20260808 17:04:19.238584 23703 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:19.238799 23703 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b0929cd4a74a77baf22d08a08b510b, State: Initialized, Role: FOLLOWER
I20260808 17:04:19.238934 23703 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:19.238998 23703 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:04:19.239038 23703 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:04:19.239084 23703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 2 FOLLOWER]: Advancing to term 3
I20260808 17:04:19.240058 23703 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:19.240147 23703 leader_election.cc:302] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7b0929cd4a74a77baf22d08a08b510b; no voters:
I20260808 17:04:19.240298 23703 leader_election.cc:288] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 17:04:19.240357 23708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 3 FOLLOWER]: Leader election won for term 3
I20260808 17:04:19.240475 23708 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [term 3 LEADER]: Becoming Leader. State: Replica: c7b0929cd4a74a77baf22d08a08b510b, State: Running, Role: LEADER
I20260808 17:04:19.240568 23708 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } }
I20260808 17:04:19.240595 23703 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:04:19.240828 23708 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7b0929cd4a74a77baf22d08a08b510b. Latest consensus state: current_term: 3 leader_uuid: "c7b0929cd4a74a77baf22d08a08b510b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } } }
I20260808 17:04:19.240906 23708 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:19.240849 23712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c7b0929cd4a74a77baf22d08a08b510b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0929cd4a74a77baf22d08a08b510b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40309 } } }
I20260808 17:04:19.241047 23712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b0929cd4a74a77baf22d08a08b510b [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:19.241761 23715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:04:19.242559 23715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:04:19.242664 23715 catalog_manager.cc:1259] Loaded cluster ID: 3063efba345d4478872cf3c91a939205
I20260808 17:04:19.242702 23715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 17:04:19.243165 23715 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdc59eab9ad google::LogMessage::Fail() at ??:0
@ 0x7fdc59eae88a google::LogMessage::SendToLog() at ??:0
@ 0x7fdc59eab4de google::LogMessage::Flush() at ??:0
@ 0x7fdc59eaca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdc61dcc0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fdc61dd58b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdc5bd790d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdc5bd6e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdc5cbd26db start_thread at ??:0
@ 0x7fdc589a961f clone at ??:0
W20260808 17:04:19.319391 23705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:19.319578 23705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:19.319612 23705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:19.320722 23705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:19.320762 23705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:19.320796 23705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:04:19.322238 23705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:33709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=34321
--tserver_master_addrs=127.30.194.62:40309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:19.322474 23705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:19.322671 23705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:19.327401 23760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:19.327450 23755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:19.327451 23763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:19.327517 23705 server_base.cc:1034] running on GCE node
I20260808 17:04:19.327955 23705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:19.328190 23705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:19.329331 23705 hybrid_clock.cc:648] HybridClock initialized: now 1786208659329311 us; error 35 us; skew 500 ppm
Aug 08 17:04:19 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208659, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:19.331612 23705 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:04:19.332036 23705 webserver.cc:460] Webserver started at http://127.30.194.1:34321/ using document root <none> and password file <none>
I20260808 17:04:19.332247 23705 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:19.332294 23705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:19.333609 23705 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:19.334265 23776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:19.334436 23705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:19.334506 23705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "b7b2bfed18ce1a027a0991ed423f6c77"
server_key_iv: "e991e3390bc9e7a37826baac055c4444"
server_key_version: "encryptionkey@0"
I20260808 17:04:19.334813 23705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:19.358124 23705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:19.360167 23705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:19.360280 23705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:19.360496 23705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:19.360919 23788 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:04:19.362769 23705 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:04:19.362814 23705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 17:04:19.362846 23705 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:04:19.363445 23705 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:04:19.363477 23705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 17:04:19.363569 23788 tablet_bootstrap.cc:492] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Bootstrap starting.
I20260808 17:04:19.370103 23705 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:33709
I20260808 17:04:19.370230 23983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:33709 every 8 connection(s)
I20260808 17:04:19.370489 23705 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:04:19.374541 23788 log.cc:824] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:19.375593 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 23705
I20260808 17:04:19.375828 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:37059
--local_ip_for_outbound_sockets=127.30.194.2
--tserver_master_addrs=127.30.194.62:40309
--webserver_port=33147
--webserver_interface=127.30.194.2
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:19.377135 23788 tablet_bootstrap.cc:492] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:04:19.377416 23788 tablet_bootstrap.cc:492] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Bootstrap complete.
I20260808 17:04:19.378496 23788 ts_tablet_manager.cc:1397] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.011s
I20260808 17:04:19.379817 23788 raft_consensus.cc:348] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:19.380151 23788 raft_consensus.cc:729] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 289bfa4ac1fc4059b99ac51c06b7c9cd, State: Initialized, Role: FOLLOWER
I20260808 17:04:19.380295 23788 consensus_queue.cc:260] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:19.380549 23788 ts_tablet_manager.cc:1428] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260808 17:04:19.429019 23988 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:04:19.371164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:59845)
0808 17:04:19.371326 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:19.371358 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:19.371781 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:19.428867 (+ 57086us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
W20260808 17:04:19.429281 23984 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:40309 (0 consecutive failures): Network error: Failed to ping master at 127.30.194.62:40309: Client connection negotiation failed: client connection to 127.30.194.62:40309: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 17:04:19.429600 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:19.429501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:34259)
0808 17:04:19.429545 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:19.429581 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 17:04:19.429932 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:19.429849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:41685)
0808 17:04:19.429886 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:19.429918 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 17:04:19.430032 23984 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:04:19.460495 23992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:19.460669 23992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:19.460691 23992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:19.461925 23992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:19.461963 23992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:19.461989 23992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:04:19.463356 23992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:37059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=33147
--tserver_master_addrs=127.30.194.62:40309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:19.463551 23992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:19.463719 23992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:19.468250 24008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:19.468248 24006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:19.468302 24005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:19.468514 23992 server_base.cc:1034] running on GCE node
I20260808 17:04:19.468703 23992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:19.468910 23992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:19.470036 23992 hybrid_clock.cc:648] HybridClock initialized: now 1786208659470018 us; error 34 us; skew 500 ppm
Aug 08 17:04:19 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208659, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:19.472323 23992 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:04:19.472743 23992 webserver.cc:460] Webserver started at http://127.30.194.2:33147/ using document root <none> and password file <none>
I20260808 17:04:19.472942 23992 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:19.472988 23992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:19.474216 23992 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:19.474711 24018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:19.474855 23992 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:19.474939 23992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "287e98c805193f6ddc58d5fd498dc6ec"
server_key_iv: "6a93ca5a5bbbf7fbf363ee4cff96395c"
server_key_version: "encryptionkey@0"
I20260808 17:04:19.475237 23992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:19.508924 23992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:19.510722 23992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:19.510866 23992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:19.511096 23992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:19.511404 23992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:19.511438 23992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:19.511468 23992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:19.511487 23992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:19.517050 23992 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:37059
I20260808 17:04:19.517138 24217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:37059 every 8 connection(s)
I20260808 17:04:19.517449 23992 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:04:19.518414 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:19.518053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:45621)
0808 17:04:19.518200 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:19.518277 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
W20260808 17:04:19.518788 24218 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:40309 (0 consecutive failures): Network error: Failed to ping master at 127.30.194.62:40309: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
I20260808 17:04:19.519034 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:19.518948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:44875)
0808 17:04:19.518987 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:19.519012 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 17:04:19.519268 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:19.519194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:54111)
0808 17:04:19.519228 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:19.519248 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 17:04:19.519356 24218 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:04:19.525193 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 23992
I20260808 17:04:19.525473 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:43667
--local_ip_for_outbound_sockets=127.30.194.3
--tserver_master_addrs=127.30.194.62:40309
--webserver_port=37951
--webserver_interface=127.30.194.3
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:45287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:19.604723 24222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:19.604912 24222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:19.604934 24222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:19.606097 24222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:19.606137 24222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:19.606161 24222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:04:19.607515 24222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:45287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:43667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=37951
--tserver_master_addrs=127.30.194.62:40309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:19.607677 24222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:19.607848 24222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:19.612332 24230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:19.612393 24231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:19.612394 24233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:19.612375 24222 server_base.cc:1034] running on GCE node
I20260808 17:04:19.612752 24222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:19.612973 24222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:19.614109 24222 hybrid_clock.cc:648] HybridClock initialized: now 1786208659614092 us; error 28 us; skew 500 ppm
Aug 08 17:04:19 dist-test-slave-2j7r krb5kdc[21842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208659, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:19.616361 24222 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:04:19.616760 24222 webserver.cc:460] Webserver started at http://127.30.194.3:37951/ using document root <none> and password file <none>
I20260808 17:04:19.616943 24222 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:19.616984 24222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:19.618162 24222 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:19.618690 24244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:19.618839 24222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:19.618913 24222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "2e8e4d77dcf04a6cb22f473bf088b54d"
format_stamp: "Formatted at 2026-08-08 17:04:17 on dist-test-slave-2j7r"
server_key: "ea3efb925323dc8a1cc203725d69446b"
server_key_iv: "2710d71d8c98efd8a414b353c3390f9d"
server_key_version: "encryptionkey@0"
I20260808 17:04:19.619189 24222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:19.629432 24222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:19.630975 24222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:19.631088 24222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:19.631266 24222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:19.631561 24222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:19.631595 24222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:19.631616 24222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:19.631639 24222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:19.639127 24222 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:43667
I20260808 17:04:19.639181 24432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:43667 every 8 connection(s)
I20260808 17:04:19.639468 24222 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:04:19.640374 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:19.640035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:44021)
0808 17:04:19.640177 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:19.640248 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260808 17:04:19.640697 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 24222
W20260808 17:04:19.640746 24433 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:40309 (0 consecutive failures): Network error: Failed to ping master at 127.30.194.62:40309: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
I20260808 17:04:19.641006 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:19.640917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:36495)
0808 17:04:19.640961 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:19.640988 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 17:04:19.641290 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:19.641229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:43311)
0808 17:04:19.641255 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:19.641275 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 17:04:19.641391 24433 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:04:20.430526 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:20.430315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:51315)
0808 17:04:20.430431 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:20.430463 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260808 17:04:20.519763 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:20.519597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:49321)
0808 17:04:20.519697 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:20.519724 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":53,"threads_started":1}
I20260808 17:04:20.641800 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:20.641633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:36631)
0808 17:04:20.641726 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:20.641757 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":54,"threads_started":1}
I20260808 17:04:20.942865 24443 raft_consensus.cc:482] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:20.942978 24443 raft_consensus.cc:504] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:20.943320 24443 leader_election.cc:288] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667), b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059)
Aug 08 17:04:20 dist-test-slave-2j7r krb5kdc[21842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208659, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.3@KRBTEST.COM
Aug 08 17:04:20 dist-test-slave-2j7r krb5kdc[21842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208659, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.2@KRBTEST.COM
I20260808 17:04:20.952705 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:20.943561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:53433 (local address 127.30.194.3:43667)
0808 17:04:20.943730 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:20.943739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:20.943766 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:04:20.944186 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:20.944188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:20.944228 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:20.944357 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:20.945108 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:20.945979 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:20.946940 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:20.947396 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:20.947496 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:20.949468 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:20.949471 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:20.949472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:20.949481 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:20.951120 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:20.951560 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:20.951563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:20.951564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:20.951672 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:20.951925 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:20.951925 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:20.951926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:20.952135 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:20.952206 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:20.952481 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 17:04:20.952595 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":62,"threads_started":1}
I20260808 17:04:20.952690 24447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:20.943435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:37059 (local address 127.30.194.1:51687)
0808 17:04:20.943536 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:20.943546 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:20.943602 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:20.944423 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:20.944426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:20.944791 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:20.944970 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:20.944976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:20.946095 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:20.946096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:20.946886 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:20.946889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:20.947564 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:20.947565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:20.947618 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:20.948125 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:20.948129 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:20.949740 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:20.951355 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:20.951356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:20.951356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:20.951570 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:20.951802 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:20.951803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:20.951803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:20.951857 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:20.952201 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:20.952208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:20.952330 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 17:04:20.952415 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 17:04:20.952591 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":36,"thread_start_us":48,"threads_started":1}
I20260808 17:04:20.952703 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:20.943562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:51687 (local address 127.30.194.2:37059)
0808 17:04:20.943724 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:20.943735 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:20.943765 (+ 30us) server_negotiation.cc:407] Connection header received
0808 17:04:20.944187 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:20.944189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:20.944236 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:20.944377 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:20.945108 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:20.945995 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:20.946957 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:20.947490 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:20.947583 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:20.949830 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:20.949833 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:20.949833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:20.949845 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:20.951275 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:20.951644 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:20.951646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:20.951646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:20.951738 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:20.951910 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:20.951911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:20.951911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:20.952135 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:20.952206 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:20.952480 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 17:04:20.952596 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":57,"threads_started":1}
I20260808 17:04:20.952793 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:20.943435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:43667 (local address 127.30.194.1:53433)
0808 17:04:20.943537 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:20.943546 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:20.943602 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:20.944421 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:20.944425 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:20.944789 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:20.944970 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:20.944976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:20.946073 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:20.946074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:20.946846 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:20.946848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:20.947478 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:20.947479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:20.947587 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:20.947993 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:20.948001 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:20.949376 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:20.951202 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:20.951209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:20.951211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:20.951492 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:20.951733 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:20.951734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:20.951735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:20.951854 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:20.952200 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:20.952207 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:20.952329 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 17:04:20.952417 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 17:04:20.952679 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":158,"thread_start_us":60,"threads_started":1}
W20260808 17:04:20.954068 23781 leader_election.cc:341] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059): Not found: Tablet not found: 460b0365ddfa4ecebf2f33d884b9e63c
W20260808 17:04:20.954155 23783 leader_election.cc:341] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667): Not found: Tablet not found: 460b0365ddfa4ecebf2f33d884b9e63c
I20260808 17:04:20.954211 23783 leader_election.cc:302] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 289bfa4ac1fc4059b99ac51c06b7c9cd; no voters: 2e8e4d77dcf04a6cb22f473bf088b54d, b9e52d94e93b4fb08ecb71ddd1e7a439
I20260808 17:04:20.954299 24443 raft_consensus.cc:2726] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:04:21.431082 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:21.430939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:35249)
0808 17:04:21.431013 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:21.431040 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 17:04:21.520323 24451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:21.520136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:36731)
0808 17:04:21.520249 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:21.520285 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260808 17:04:21.642684 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:21.642149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:35171)
0808 17:04:21.642601 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:21.642640 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":387,"threads_started":1}
I20260808 17:04:22.431653 24455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:22.431450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:46609)
0808 17:04:22.431569 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:22.431607 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260808 17:04:22.520855 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:22.520653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:47663)
0808 17:04:22.520781 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:22.520816 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260808 17:04:22.643250 24457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:22.643058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:46909)
0808 17:04:22.643183 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:22.643213 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260808 17:04:23.066862 24458 raft_consensus.cc:482] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:23.066958 24458 raft_consensus.cc:504] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:23.067095 24458 leader_election.cc:288] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667), b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059)
W20260808 17:04:23.067494 23783 leader_election.cc:341] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667): Not found: Tablet not found: 460b0365ddfa4ecebf2f33d884b9e63c
W20260808 17:04:23.067589 23781 leader_election.cc:341] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059): Not found: Tablet not found: 460b0365ddfa4ecebf2f33d884b9e63c
I20260808 17:04:23.067636 23781 leader_election.cc:302] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 289bfa4ac1fc4059b99ac51c06b7c9cd; no voters: 2e8e4d77dcf04a6cb22f473bf088b54d, b9e52d94e93b4fb08ecb71ddd1e7a439
I20260808 17:04:23.067718 24458 raft_consensus.cc:2726] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:04:23.432193 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:23.431995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:44149)
0808 17:04:23.432112 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:23.432146 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260808 17:04:23.521447 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:23.521254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:41041)
0808 17:04:23.521364 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:23.521395 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260808 17:04:23.643791 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:23.643603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:57965)
0808 17:04:23.643721 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:23.643753 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260808 17:04:24.432787 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:24.432595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:52303)
0808 17:04:24.432709 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:24.432743 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260808 17:04:24.521968 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:24.521782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:38139)
0808 17:04:24.521890 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:24.521921 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260808 17:04:24.644340 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:24.644132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:50819)
0808 17:04:24.644269 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:24.644301 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260808 17:04:25.286566 24470 raft_consensus.cc:482] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:25.286662 24470 raft_consensus.cc:504] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:25.286798 24470 leader_election.cc:288] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667), b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059)
W20260808 17:04:25.287142 23783 leader_election.cc:341] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667): Not found: Tablet not found: 460b0365ddfa4ecebf2f33d884b9e63c
W20260808 17:04:25.287231 23781 leader_election.cc:341] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059): Not found: Tablet not found: 460b0365ddfa4ecebf2f33d884b9e63c
I20260808 17:04:25.287292 23781 leader_election.cc:302] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 289bfa4ac1fc4059b99ac51c06b7c9cd; no voters: 2e8e4d77dcf04a6cb22f473bf088b54d, b9e52d94e93b4fb08ecb71ddd1e7a439
I20260808 17:04:25.287388 24470 raft_consensus.cc:2726] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:04:25.433380 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:25.433186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:52149)
0808 17:04:25.433295 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:25.433329 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260808 17:04:25.522523 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:25.522325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:37779)
0808 17:04:25.522451 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:25.522484 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260808 17:04:25.644907 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:25.644723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:46247)
0808 17:04:25.644835 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:25.644868 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260808 17:04:26.433969 24475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:26.433769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:47431)
0808 17:04:26.433888 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:26.433922 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260808 17:04:26.523099 24476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:26.522902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:53423)
0808 17:04:26.523016 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:26.523053 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260808 17:04:26.645504 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:26.645299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:36467)
0808 17:04:26.645432 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:26.645465 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260808 17:04:27.434588 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:27.434372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:36703)
0808 17:04:27.434497 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:27.434538 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260808 17:04:27.523675 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:27.523445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:40229)
0808 17:04:27.523592 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:27.523631 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260808 17:04:27.646164 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:27.645956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:37697)
0808 17:04:27.646074 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:27.646115 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260808 17:04:28.435199 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:28.434994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:36113)
0808 17:04:28.435115 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:28.435152 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260808 17:04:28.524227 24482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:28.524044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:46783)
0808 17:04:28.524155 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:28.524189 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260808 17:04:28.646755 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:28.646583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:53107)
0808 17:04:28.646686 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:28.646718 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":58,"threads_started":1}
I20260808 17:04:29.237722 24486 raft_consensus.cc:482] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:29.237816 24486 raft_consensus.cc:504] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:29.238011 24486 leader_election.cc:288] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667), b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059)
W20260808 17:04:29.238402 23781 leader_election.cc:341] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059): Not found: Tablet not found: 460b0365ddfa4ecebf2f33d884b9e63c
W20260808 17:04:29.238481 23783 leader_election.cc:341] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667): Not found: Tablet not found: 460b0365ddfa4ecebf2f33d884b9e63c
I20260808 17:04:29.238528 23783 leader_election.cc:302] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 289bfa4ac1fc4059b99ac51c06b7c9cd; no voters: 2e8e4d77dcf04a6cb22f473bf088b54d, b9e52d94e93b4fb08ecb71ddd1e7a439
I20260808 17:04:29.238620 24486 raft_consensus.cc:2726] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:04:29.435763 24489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:29.435577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:46597)
0808 17:04:29.435687 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:29.435716 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260808 17:04:29.524793 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:29.524596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:35769)
0808 17:04:29.524721 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:29.524754 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260808 17:04:29.647295 24495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:29.647122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:51385)
0808 17:04:29.647229 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:29.647260 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260808 17:04:30.436368 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:30.436144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:51277)
0808 17:04:30.436256 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:30.436293 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260808 17:04:30.525379 24499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:30.525176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:42691)
0808 17:04:30.525301 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:30.525333 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260808 17:04:30.647862 24500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:30.647677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:56329)
0808 17:04:30.647792 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:30.647822 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260808 17:04:31.436976 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:31.436778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:42467)
0808 17:04:31.436894 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:31.436927 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260808 17:04:31.525911 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:31.525725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:51197)
0808 17:04:31.525840 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:31.525871 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260808 17:04:31.648394 24506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:31.648204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:51393)
0808 17:04:31.648316 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:31.648349 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260808 17:04:32.437566 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:32.437338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:38667)
0808 17:04:32.437484 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:32.437517 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260808 17:04:32.526499 24508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:32.526282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:33745)
0808 17:04:32.526415 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:32.526460 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260808 17:04:32.648936 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:32.648745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:41749)
0808 17:04:32.648862 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:32.648896 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260808 17:04:33.096660 24511 raft_consensus.cc:482] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:33.096750 24511 raft_consensus.cc:504] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e8e4d77dcf04a6cb22f473bf088b54d" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43667 } } peers { permanent_uuid: "b9e52d94e93b4fb08ecb71ddd1e7a439" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37059 } } peers { permanent_uuid: "289bfa4ac1fc4059b99ac51c06b7c9cd" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 33709 } }
I20260808 17:04:33.096886 24511 leader_election.cc:288] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667), b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059)
W20260808 17:04:33.097265 23783 leader_election.cc:341] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e8e4d77dcf04a6cb22f473bf088b54d (127.30.194.3:43667): Not found: Tablet not found: 460b0365ddfa4ecebf2f33d884b9e63c
W20260808 17:04:33.097347 23781 leader_election.cc:341] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9e52d94e93b4fb08ecb71ddd1e7a439 (127.30.194.2:37059): Not found: Tablet not found: 460b0365ddfa4ecebf2f33d884b9e63c
I20260808 17:04:33.097397 23781 leader_election.cc:302] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 289bfa4ac1fc4059b99ac51c06b7c9cd; no voters: 2e8e4d77dcf04a6cb22f473bf088b54d, b9e52d94e93b4fb08ecb71ddd1e7a439
I20260808 17:04:33.097463 24511 raft_consensus.cc:2726] T 460b0365ddfa4ecebf2f33d884b9e63c P 289bfa4ac1fc4059b99ac51c06b7c9cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:04:33.438156 24513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:33.437920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:42765)
0808 17:04:33.438061 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:33.438098 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260808 17:04:33.527061 24514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:33.526841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:42627)
0808 17:04:33.526989 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:33.527021 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260808 17:04:33.649466 24516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:33.649268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:39981)
0808 17:04:33.649385 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:33.649415 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260808 17:04:34.438825 24517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:34.438637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.1:55015)
0808 17:04:34.438741 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:34.438772 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
I20260808 17:04:34.527688 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:34.527496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.2:41745)
0808 17:04:34.527618 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:34.527649 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260808 17:04:34.647504 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 23705
I20260808 17:04:34.656893 24520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:34.656701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40309 (local address 127.30.194.3:36479)
0808 17:04:34.656818 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:34.656851 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260808 17:04:34.661218 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 23992
I20260808 17:04:34.669478 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 24222
2026-08-08T17:04:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18976 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:04:34 dist-test-slave-2j7r krb5kdc[24523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:04:34 dist-test-slave-2j7r krb5kdc[24523](info): set up 2 sockets
Aug 08 17:04:34 dist-test-slave-2j7r krb5kdc[24523](info): commencing operation
krb5kdc: starting...
W20260808 17:04:35.850131 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:04:35 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:04:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:04:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:04:35.960760 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43273
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:36.040092 24544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:36.040283 24544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:36.040320 24544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:36.041435 24544 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 17:04:36.041471 24544 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 17:04:36.041488 24544 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 17:04:36.041503 24544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:36.041518 24544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:36.041536 24544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:36.041559 24544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:36.041576 24544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:36.043004 24544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:36.043231 24544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:36.043428 24544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:36.048180 24556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:36.048218 24544 server_base.cc:1034] running on GCE node
W20260808 17:04:36.048246 24557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:36.048259 24561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:36.048645 24544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:36.048900 24544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:36.050032 24544 hybrid_clock.cc:648] HybridClock initialized: now 1786208676050018 us; error 33 us; skew 500 ppm
Aug 08 17:04:36 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208676, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:36.056835 24544 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:04:36.057291 24544 webserver.cc:460] Webserver started at http://127.30.194.62:37161/ using document root <none> and password file <none>
I20260808 17:04:36.057492 24544 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:36.057538 24544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:36.057657 24544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:36.058635 24544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "d62ea5d867294f23aa04748e29320bea"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "b7a129aaeeaa81f9aae1a9af61ef34b7"
server_key_iv: "e1f4f91d9263c2dc0f98dc209ec23f30"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.058951 24544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "d62ea5d867294f23aa04748e29320bea"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "b7a129aaeeaa81f9aae1a9af61ef34b7"
server_key_iv: "e1f4f91d9263c2dc0f98dc209ec23f30"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.060133 24544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:04:36.060762 24573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:36.060914 24544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:36.060997 24544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "d62ea5d867294f23aa04748e29320bea"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "b7a129aaeeaa81f9aae1a9af61ef34b7"
server_key_iv: "e1f4f91d9263c2dc0f98dc209ec23f30"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.061056 24544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:36.070349 24544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:36.071849 24544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:36.071950 24544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:36.075660 24544 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:43273
I20260808 17:04:36.075695 24664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:43273 every 8 connection(s)
I20260808 17:04:36.075971 24544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:04:36.076749 24665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:36.078825 24665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Bootstrap starting.
I20260808 17:04:36.079340 24665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:36.079568 24665 log.cc:824] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:36.080128 24665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: No bootstrap required, opened a new log
I20260808 17:04:36.081672 24665 raft_consensus.cc:348] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:04:36.081794 24665 raft_consensus.cc:374] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:36.081826 24665 raft_consensus.cc:729] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ea5d867294f23aa04748e29320bea, State: Initialized, Role: FOLLOWER
I20260808 17:04:36.081923 24665 consensus_queue.cc:260] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:04:36.081971 24665 raft_consensus.cc:388] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:04:36.081990 24665 raft_consensus.cc:482] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:04:36.082018 24665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:36.082463 24665 raft_consensus.cc:504] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:04:36.082540 24665 leader_election.cc:302] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d62ea5d867294f23aa04748e29320bea; no voters:
I20260808 17:04:36.082656 24665 leader_election.cc:288] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:04:36.082701 24670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:04:36.082857 24665 sys_catalog.cc:564] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:04:36.082892 24670 raft_consensus.cc:686] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 1 LEADER]: Becoming Leader. State: Replica: d62ea5d867294f23aa04748e29320bea, State: Running, Role: LEADER
I20260808 17:04:36.083014 24670 consensus_queue.cc:237] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:04:36.083387 24674 sys_catalog.cc:455] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d62ea5d867294f23aa04748e29320bea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } } }
I20260808 17:04:36.083369 24675 sys_catalog.cc:455] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: SysCatalogTable state changed. Reason: New leader d62ea5d867294f23aa04748e29320bea. Latest consensus state: current_term: 1 leader_uuid: "d62ea5d867294f23aa04748e29320bea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } } }
I20260808 17:04:36.083483 24674 sys_catalog.cc:458] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:36.083490 24675 sys_catalog.cc:458] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:36.083885 24680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:04:36.084353 24680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:04:36.084702 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 24544
I20260808 17:04:36.084797 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:04:36.084980 31496 external_mini_cluster.cc:1428] Setting key 9d8b0380c480abd380cb83854bc51e9d
I20260808 17:04:36.086467 24680 catalog_manager.cc:1347] Generated new cluster ID: e25e63b3dc7e43c9bf8a5ccf24aeba4b
I20260808 17:04:36.086517 24680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:04:36 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:36.094316 24696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.085404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.30.194.62:43273)
0808 17:04:36.085696 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.085704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.085736 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:04:36.085850 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.085857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.085912 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.086046 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:36.086807 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.087704 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.088426 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.088852 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.088945 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.091101 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:36.091105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:36.091105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:36.091118 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:36.092510 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:36.092826 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:36.092828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:36.092829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:36.092929 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:36.093172 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:36.093173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:36.093174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:36.093481 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:36.093999 (+ 518us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.094057 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.094156 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":78,"threads_started":1}
I20260808 17:04:36.103272 24680 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:04:36.103717 24680 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:04:36.109622 24680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Generated new TSK 0
I20260808 17:04:36.109777 24680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:04:36.137176 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43273
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:36.219545 24701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:36.219732 24701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:36.219753 24701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:36.220858 24701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:36.220897 24701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:36.220925 24701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:04:36.222398 24701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:36.222616 24701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:36.222838 24701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:36.227383 24708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:36.227396 24714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:36.227411 24718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:36.227515 24701 server_base.cc:1034] running on GCE node
I20260808 17:04:36.227782 24701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:36.228020 24701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:36.229138 24701 hybrid_clock.cc:648] HybridClock initialized: now 1786208676229122 us; error 28 us; skew 500 ppm
Aug 08 17:04:36 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208676, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:36.231438 24701 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:04:36.231791 24701 webserver.cc:460] Webserver started at http://127.30.194.1:43569/ using document root <none> and password file <none>
I20260808 17:04:36.231971 24701 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:36.232008 24701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:36.232096 24701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:36.233019 24701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "bb3686e4e6a845fbafa65fdc1b9be694"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "8eb10382aaaf321b1c504c8f469a0ac2"
server_key_iv: "4ff4cbd7b0633545cb167c81cc86a239"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.233350 24701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "bb3686e4e6a845fbafa65fdc1b9be694"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "8eb10382aaaf321b1c504c8f469a0ac2"
server_key_iv: "4ff4cbd7b0633545cb167c81cc86a239"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.234480 24701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:36.235093 24732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:36.235232 24701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:36.235316 24701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "bb3686e4e6a845fbafa65fdc1b9be694"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "8eb10382aaaf321b1c504c8f469a0ac2"
server_key_iv: "4ff4cbd7b0633545cb167c81cc86a239"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.235368 24701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:36.246007 24701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:36.247953 24701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:36.248054 24701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:36.248206 24701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:36.248438 24701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:36.248471 24701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:36.248495 24701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:36.248507 24701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:36.253692 24701 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:41241
I20260808 17:04:36.253757 24911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:41241 every 8 connection(s)
I20260808 17:04:36.254007 24701 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:04:36 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208676, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:36.261909 24696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.254645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:48763 (local address 127.30.194.62:43273)
0808 17:04:36.254687 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.254689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.254856 (+ 167us) server_negotiation.cc:407] Connection header received
0808 17:04:36.255303 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.255305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.255322 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.255373 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:36.256236 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.256732 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.257713 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.257931 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.257988 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.259846 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:36.259848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:36.259848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:36.259855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:36.260878 (+ 1023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:36.261244 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:36.261245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:36.261245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:36.261291 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:36.261467 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:36.261468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:36.261468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:36.261555 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:36.261597 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.261786 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.261847 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:04:36.262066 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.254610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:48763)
0808 17:04:36.254741 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:36.254776 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:36.255204 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:36.255436 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:36.255440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:36.255959 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:36.256134 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.256139 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.256816 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.256817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.257622 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.257624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.258004 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.258005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.258115 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.258515 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:36.258524 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:36.259769 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:36.260942 (+ 1173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:36.260949 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:36.260950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:36.261196 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:36.261341 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:36.261341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:36.261341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:36.261431 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:36.261594 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:36.261596 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:36.261693 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 17:04:36.261764 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:04:36.261882 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":63,"threads_started":1}
I20260808 17:04:36.262526 24913 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43273
I20260808 17:04:36.262602 24913 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:36.262732 24913 heartbeater.cc:511] Master 127.30.194.62:43273 requested a full tablet report, sending...
I20260808 17:04:36.263185 24607 ts_manager.cc:194] Registered new tserver with Master: bb3686e4e6a845fbafa65fdc1b9be694 (127.30.194.1:41241)
I20260808 17:04:36.263383 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 24701
I20260808 17:04:36.263490 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:04:36.263656 31496 external_mini_cluster.cc:1428] Setting key a49b29a880851831367a66a56cb020e8
I20260808 17:04:36.263885 24607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:48763
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:04:36.287231 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43273
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:36.364643 24918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:36.364820 24918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:36.364841 24918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:36.365996 24918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:36.366035 24918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:36.366065 24918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:04:36.367456 24918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:36.367655 24918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:36.367846 24918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:04:36.372390 24918 server_base.cc:1034] running on GCE node
W20260808 17:04:36.372325 24933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:36.372329 24924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:36.372354 24930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:36.372685 24918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:36.372879 24918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:36.374025 24918 hybrid_clock.cc:648] HybridClock initialized: now 1786208676374015 us; error 27 us; skew 500 ppm
Aug 08 17:04:36 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208676, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:36.376256 24918 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:04:36.376649 24918 webserver.cc:460] Webserver started at http://127.30.194.2:32931/ using document root <none> and password file <none>
I20260808 17:04:36.376818 24918 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:36.376855 24918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:36.376940 24918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:36.377857 24918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "cc9ba4a566624e7c8f32616bf133b301"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "1d9a92b7a84d6ca154485536d0d6e6be"
server_key_iv: "1d1b6825efe0d95feada0a7bd13146c8"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.378123 24918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "cc9ba4a566624e7c8f32616bf133b301"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "1d9a92b7a84d6ca154485536d0d6e6be"
server_key_iv: "1d1b6825efe0d95feada0a7bd13146c8"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.379215 24918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:04:36.379781 24942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:36.379932 24918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:04:36.380013 24918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "cc9ba4a566624e7c8f32616bf133b301"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "1d9a92b7a84d6ca154485536d0d6e6be"
server_key_iv: "1d1b6825efe0d95feada0a7bd13146c8"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.380064 24918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:36.401561 24918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:36.403100 24918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:36.403203 24918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:36.403379 24918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:36.403656 24918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:36.403690 24918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:36.403712 24918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:36.403734 24918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:36.409013 24918 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:39323
I20260808 17:04:36.409103 25115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:39323 every 8 connection(s)
I20260808 17:04:36.409370 24918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:04:36.411307 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 24918
I20260808 17:04:36.411424 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:04:36.411604 31496 external_mini_cluster.cc:1428] Setting key 37b0b89d8267468b7e627f1cfafccc94
Aug 08 17:04:36 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208676, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:36.418787 24696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.410059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:55973 (local address 127.30.194.62:43273)
0808 17:04:36.410135 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.410138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.410306 (+ 168us) server_negotiation.cc:407] Connection header received
0808 17:04:36.410751 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.410752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.410762 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.410814 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:36.411788 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.412464 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.413532 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.413793 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.413867 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.416168 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:36.416169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:36.416169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:36.416176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:36.417475 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:36.417912 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:36.417913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:36.417913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:36.417958 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:36.418202 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:36.418202 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:36.418202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:36.418319 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:36.418365 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.418660 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.418724 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 17:04:36.419026 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.410026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:55973)
0808 17:04:36.410192 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:36.410227 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:36.410678 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:36.410882 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:36.410886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:36.411471 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:36.411694 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.411700 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.412551 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.412552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.413457 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.413460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.413886 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.413887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.414025 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.414521 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:36.414532 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:36.416075 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:36.417545 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:36.417552 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:36.417554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:36.417841 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:36.418009 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:36.418010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:36.418010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:36.418146 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:36.418374 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:36.418381 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:36.418534 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 17:04:36.418654 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 17:04:36.418813 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260808 17:04:36.419523 25116 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43273
I20260808 17:04:36.419608 25116 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:36.419778 25116 heartbeater.cc:511] Master 127.30.194.62:43273 requested a full tablet report, sending...
I20260808 17:04:36.420140 24606 ts_manager.cc:194] Registered new tserver with Master: cc9ba4a566624e7c8f32616bf133b301 (127.30.194.2:39323)
I20260808 17:04:36.420471 24606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:55973
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:04:36.437541 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43273
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:36.518085 25123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:36.518278 25123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:36.518302 25123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:36.519372 25123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:36.519409 25123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:36.519436 25123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:04:36.520798 25123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:36.520979 25123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:36.521206 25123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:36.525892 25131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:36.526019 25123 server_base.cc:1034] running on GCE node
W20260808 17:04:36.525902 25136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:36.526116 25130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:36.526312 25123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:36.526538 25123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:36.527678 25123 hybrid_clock.cc:648] HybridClock initialized: now 1786208676527659 us; error 37 us; skew 500 ppm
Aug 08 17:04:36 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208676, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:36.530184 25123 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:04:36.530592 25123 webserver.cc:460] Webserver started at http://127.30.194.3:34753/ using document root <none> and password file <none>
I20260808 17:04:36.530799 25123 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:36.530845 25123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:36.530957 25123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:04:36.531941 25123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "7766a88f15b3404a911286aca42da04c"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "e76b0d8aa22c4d2cb1aaf7c12c58b728"
server_key_iv: "134339300bf60c965c0eeb7f346b33a8"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.532232 25123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "7766a88f15b3404a911286aca42da04c"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "e76b0d8aa22c4d2cb1aaf7c12c58b728"
server_key_iv: "134339300bf60c965c0eeb7f346b33a8"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.533433 25123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:04:36.534049 25144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:36.534193 25123 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:04:36.534274 25123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "7766a88f15b3404a911286aca42da04c"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "e76b0d8aa22c4d2cb1aaf7c12c58b728"
server_key_iv: "134339300bf60c965c0eeb7f346b33a8"
server_key_version: "encryptionkey@0"
I20260808 17:04:36.534339 25123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:36.542215 25123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:36.543679 25123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:36.543799 25123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:36.543979 25123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:36.544234 25123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:36.544265 25123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:36.544291 25123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:36.544322 25123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:36.549857 25123 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:42443
I20260808 17:04:36.549885 25317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:42443 every 8 connection(s)
I20260808 17:04:36.550177 25123 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:04:36.552078 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 25123
I20260808 17:04:36.552171 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:04:36.552345 31496 external_mini_cluster.cc:1428] Setting key cd4127a0880667069b80ddeb06729d02
Aug 08 17:04:36 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208676, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:36.558320 24696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.550830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:44049 (local address 127.30.194.62:43273)
0808 17:04:36.550893 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.550895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.551051 (+ 156us) server_negotiation.cc:407] Connection header received
0808 17:04:36.551495 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.551497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.551510 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.551557 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:36.552441 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.553139 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.554056 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.554251 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.554310 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.556115 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:36.556116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:36.556117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:36.556124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:36.557163 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:36.557493 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:36.557495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:36.557495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:36.557552 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:36.557748 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:36.557749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:36.557749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:36.557857 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:36.557913 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.558147 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.558222 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:04:36.558442 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.550778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:44049)
0808 17:04:36.550943 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:36.550979 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:36.551401 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:36.551630 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:36.551635 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:36.552166 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:36.552366 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.552371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.553213 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.553215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.553975 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.553977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.554323 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.554324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.554446 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.554829 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:36.554836 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:36.556032 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:36.557223 (+ 1191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:36.557230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:36.557232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:36.557429 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:36.557603 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:36.557603 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:36.557604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:36.557694 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:36.557910 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:36.557916 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:36.558035 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 17:04:36.558117 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 17:04:36.558251 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260808 17:04:36.558831 25318 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43273
I20260808 17:04:36.558911 25318 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:36.559063 25318 heartbeater.cc:511] Master 127.30.194.62:43273 requested a full tablet report, sending...
I20260808 17:04:36.559396 24606 ts_manager.cc:194] Registered new tserver with Master: 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443)
I20260808 17:04:36.559753 24606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:44049
I20260808 17:04:36.563686 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:04:36.570039 24696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.564914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46668 (local address 127.30.194.62:43273)
0808 17:04:36.564953 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.564955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.565058 (+ 103us) server_negotiation.cc:407] Connection header received
0808 17:04:36.565135 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.565136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.565146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.565203 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:36.565723 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.566159 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.566979 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.567147 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.567187 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.568138 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:36.568139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:36.568139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:36.568145 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:36.569191 (+ 1046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:36.569488 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:36.569489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:36.569489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:36.569533 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:36.569666 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:36.569666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:36.569666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:36.569783 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:36.569850 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.569931 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.569981 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:04:36.571408 24606 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:04:36.572815 24606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:04:36.578557 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.574928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:42443 (local address 127.0.0.1:41686)
0808 17:04:36.575231 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:36.575240 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:36.575311 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:36.575514 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:36.575516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:36.575518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:36.575564 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.575565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.576382 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.576383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.577590 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.577593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.578285 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.578287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.578363 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.578376 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:04:36.578439 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 17:04:36.578488 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":74,"threads_started":1}
I20260808 17:04:36.578567 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.575095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41686 (local address 127.30.194.3:42443)
0808 17:04:36.575230 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.575234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.575300 (+ 66us) server_negotiation.cc:407] Connection header received
0808 17:04:36.575368 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.575370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.575383 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.575451 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:36.575631 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.576295 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.577678 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.578212 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.578274 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.578340 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.578457 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.578490 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260808 17:04:36.578851 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.574817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:41241 (local address 127.0.0.1:35042)
0808 17:04:36.575044 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:36.575054 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:36.575106 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:36.575285 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:36.575288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:36.575290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:36.575347 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.575349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.576596 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.576597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.577658 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.577660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.578618 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.578620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.578681 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.578684 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:04:36.578731 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:04:36.578768 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":32,"threads_started":1}
I20260808 17:04:36.579067 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.574938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35042 (local address 127.30.194.1:41241)
0808 17:04:36.575134 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.575139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.575150 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:04:36.575190 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.575192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.575206 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.575274 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:36.575427 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.576498 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.577733 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.578456 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.578525 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.578582 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.578958 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.578998 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
I20260808 17:04:36.578850 25330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.574633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:39323 (local address 127.0.0.1:38336)
0808 17:04:36.574937 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:36.574976 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:36.575047 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:36.575628 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:36.575629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:36.575630 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:36.575660 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.575661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.576806 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.576814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.577766 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.577768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.578647 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.578647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.578697 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.578700 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:04:36.578741 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 17:04:36.578782 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":47,"threads_started":1}
I20260808 17:04:36.579370 25220 tablet_service.cc:1467] Processing CreateTablet for tablet d7d87d75006c400e975a8a90ebceeec2 (DEFAULT_TABLE table=test-table [id=0f40fb4a45ed4bb4a33b4b28f2e59de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:36.579643 25220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7d87d75006c400e975a8a90ebceeec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:36.579691 24810 tablet_service.cc:1467] Processing CreateTablet for tablet d7d87d75006c400e975a8a90ebceeec2 (DEFAULT_TABLE table=test-table [id=0f40fb4a45ed4bb4a33b4b28f2e59de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:36.579998 24810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7d87d75006c400e975a8a90ebceeec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:36.581986 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.574762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38336 (local address 127.30.194.2:39323)
0808 17:04:36.574932 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.574936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.575373 (+ 437us) server_negotiation.cc:407] Connection header received
0808 17:04:36.575479 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.575482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.575495 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.575558 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:36.575721 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.576719 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.577839 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.578519 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.578574 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.578618 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.578892 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.578933 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260808 17:04:36.582029 25337 tablet_bootstrap.cc:492] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: Bootstrap starting.
I20260808 17:04:36.582543 25338 tablet_bootstrap.cc:492] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Bootstrap starting.
I20260808 17:04:36.582768 25337 tablet_bootstrap.cc:654] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:36.582736 25011 tablet_service.cc:1467] Processing CreateTablet for tablet d7d87d75006c400e975a8a90ebceeec2 (DEFAULT_TABLE table=test-table [id=0f40fb4a45ed4bb4a33b4b28f2e59de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:36.583087 25011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7d87d75006c400e975a8a90ebceeec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:36.583242 25338 tablet_bootstrap.cc:654] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:36.583269 25337 log.cc:824] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:36.583801 25338 log.cc:824] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:36.583926 25337 tablet_bootstrap.cc:492] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: No bootstrap required, opened a new log
I20260808 17:04:36.583997 25337 ts_tablet_manager.cc:1397] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:04:36.584522 25338 tablet_bootstrap.cc:492] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: No bootstrap required, opened a new log
I20260808 17:04:36.584591 25338 ts_tablet_manager.cc:1397] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:04:36.585827 25343 tablet_bootstrap.cc:492] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: Bootstrap starting.
I20260808 17:04:36.586200 25337 raft_consensus.cc:348] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:36.586393 25343 tablet_bootstrap.cc:654] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:36.586391 25337 raft_consensus.cc:374] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:36.586429 25337 raft_consensus.cc:729] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7766a88f15b3404a911286aca42da04c, State: Initialized, Role: FOLLOWER
I20260808 17:04:36.586599 25337 consensus_queue.cc:260] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:36.586527 25338 raft_consensus.cc:348] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:36.586684 25338 raft_consensus.cc:374] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:36.586755 25338 raft_consensus.cc:729] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb3686e4e6a845fbafa65fdc1b9be694, State: Initialized, Role: FOLLOWER
I20260808 17:04:36.586894 25343 log.cc:824] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:36.586925 25318 heartbeater.cc:503] Master 127.30.194.62:43273 was elected leader, sending a full tablet report...
I20260808 17:04:36.586939 25338 consensus_queue.cc:260] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:36.586871 25337 ts_tablet_manager.cc:1428] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:04:36.587191 25338 ts_tablet_manager.cc:1428] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 17:04:36.587316 24913 heartbeater.cc:503] Master 127.30.194.62:43273 was elected leader, sending a full tablet report...
I20260808 17:04:36.587559 25343 tablet_bootstrap.cc:492] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: No bootstrap required, opened a new log
I20260808 17:04:36.587657 25343 ts_tablet_manager.cc:1397] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:04:36.589047 25343 raft_consensus.cc:348] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:36.589206 25343 raft_consensus.cc:374] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:36.589249 25343 raft_consensus.cc:729] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9ba4a566624e7c8f32616bf133b301, State: Initialized, Role: FOLLOWER
I20260808 17:04:36.589385 25343 consensus_queue.cc:260] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:36.589619 25343 ts_tablet_manager.cc:1428] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:04:36.589625 25116 heartbeater.cc:503] Master 127.30.194.62:43273 was elected leader, sending a full tablet report...
I20260808 17:04:36.609163 25349 raft_consensus.cc:482] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:36.609230 25349 raft_consensus.cc:504] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:36.609464 25349 leader_election.cc:288] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc9ba4a566624e7c8f32616bf133b301 (127.30.194.2:39323), 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443)
I20260808 17:04:36.611996 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.609594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:39323 (local address 127.30.194.1:51429)
0808 17:04:36.609636 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:36.609645 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:36.609680 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:36.609814 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:36.609815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:36.609817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:36.609868 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.609869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.610522 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.610523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.611253 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.611255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.611790 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.611791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.611865 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.611869 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:36.611917 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 17:04:36.611950 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":3}
I20260808 17:04:36.612015 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.609650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:51429 (local address 127.30.194.2:39323)
0808 17:04:36.609683 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.609685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.609694 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:04:36.609719 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.609720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.609728 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.609788 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:36.609948 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.610455 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.611334 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.611725 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.611782 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.611788 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.611926 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.611957 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 17:04:36.612162 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.609751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:46567 (local address 127.30.194.3:42443)
0808 17:04:36.609792 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.609794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.609938 (+ 144us) server_negotiation.cc:407] Connection header received
0808 17:04:36.609963 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.609964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.609972 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.610019 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:36.610140 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.610577 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.611456 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.611821 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.611886 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.611894 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.612077 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.612114 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:04:36.612347 25353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.609680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:42443 (local address 127.30.194.1:46567)
0808 17:04:36.609866 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:36.609875 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:36.609926 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:36.610022 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:36.610022 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:36.610024 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:36.610083 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.610084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.610647 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.610648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.611377 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:36.611379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.611893 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.611894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:36.611964 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.611968 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:36.612226 (+ 258us) client_negotiation.cc:240] Negotiation successful
0808 17:04:36.612277 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":84,"threads_started":1}
I20260808 17:04:36.612335 25040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7d87d75006c400e975a8a90ebceeec2" candidate_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9ba4a566624e7c8f32616bf133b301" is_pre_election: true
I20260808 17:04:36.612438 25040 raft_consensus.cc:2454] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb3686e4e6a845fbafa65fdc1b9be694 in term 0.
I20260808 17:04:36.612610 25249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7d87d75006c400e975a8a90ebceeec2" candidate_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7766a88f15b3404a911286aca42da04c" is_pre_election: true
I20260808 17:04:36.612627 24736 leader_election.cc:302] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb3686e4e6a845fbafa65fdc1b9be694, cc9ba4a566624e7c8f32616bf133b301; no voters:
I20260808 17:04:36.612695 25249 raft_consensus.cc:2454] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb3686e4e6a845fbafa65fdc1b9be694 in term 0.
I20260808 17:04:36.612800 25349 raft_consensus.cc:2781] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:04:36.612865 25349 raft_consensus.cc:482] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:04:36.612890 25349 raft_consensus.cc:3037] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:36.613559 25349 raft_consensus.cc:504] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:36.613674 25349 leader_election.cc:288] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 1 election: Requested vote from peers cc9ba4a566624e7c8f32616bf133b301 (127.30.194.2:39323), 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443)
I20260808 17:04:36.613801 25040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7d87d75006c400e975a8a90ebceeec2" candidate_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9ba4a566624e7c8f32616bf133b301"
I20260808 17:04:36.613842 25249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7d87d75006c400e975a8a90ebceeec2" candidate_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7766a88f15b3404a911286aca42da04c"
I20260808 17:04:36.613869 25040 raft_consensus.cc:3037] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:36.613896 25249 raft_consensus.cc:3037] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:36.614580 25040 raft_consensus.cc:2454] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb3686e4e6a845fbafa65fdc1b9be694 in term 1.
I20260808 17:04:36.614609 25249 raft_consensus.cc:2454] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb3686e4e6a845fbafa65fdc1b9be694 in term 1.
I20260808 17:04:36.614719 24736 leader_election.cc:302] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7766a88f15b3404a911286aca42da04c, bb3686e4e6a845fbafa65fdc1b9be694; no voters:
I20260808 17:04:36.614816 25349 raft_consensus.cc:2781] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:04:36.614929 25349 raft_consensus.cc:686] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 LEADER]: Becoming Leader. State: Replica: bb3686e4e6a845fbafa65fdc1b9be694, State: Running, Role: LEADER
I20260808 17:04:36.614995 25349 consensus_queue.cc:237] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:36.615681 24606 catalog_manager.cc:5495] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb3686e4e6a845fbafa65fdc1b9be694 (127.30.194.1). New cstate: current_term: 1 leader_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } health_report { overall_health: UNKNOWN } } }
I20260808 17:04:36.625638 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.623772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33036 (local address 127.30.194.1:41241)
0808 17:04:36.623806 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:36.623808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:36.623831 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:04:36.623882 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:36.623883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:36.623896 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:36.623950 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:04:36.624024 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.624428 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.625066 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:36.625287 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:36.625338 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:36.625431 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:04:36.625506 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:04:36.625551 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:36.625569 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 17:04:36.625582 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:04:36.626984 25249 raft_consensus.cc:1261] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c [term 1 FOLLOWER]: Refusing update from remote peer bb3686e4e6a845fbafa65fdc1b9be694: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:36.627002 25040 raft_consensus.cc:1261] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Refusing update from remote peer bb3686e4e6a845fbafa65fdc1b9be694: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:36.627235 25349 consensus_queue.cc:1035] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:36.627345 25356 consensus_queue.cc:1035] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:36.629714 25361 mvcc.cc:205] Tried to move back new op lower bound from 7316310739462131712 to 7316310739415547904. Current Snapshot: MvccSnapshot[applied={T|T < 7316310739462131712}]
I20260808 17:04:36.631767 24607 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:04:36.631841 24607 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:04:36.632417 24607 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Sending DeleteTablet for 3 replicas of tablet d7d87d75006c400e975a8a90ebceeec2
I20260808 17:04:36.632719 25011 tablet_service.cc:1514] Processing DeleteTablet for tablet d7d87d75006c400e975a8a90ebceeec2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:36 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:38336
I20260808 17:04:36.632758 24810 tablet_service.cc:1514] Processing DeleteTablet for tablet d7d87d75006c400e975a8a90ebceeec2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:36 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:35042
I20260808 17:04:36.632745 25220 tablet_service.cc:1514] Processing DeleteTablet for tablet d7d87d75006c400e975a8a90ebceeec2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:36 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:41686
I20260808 17:04:36.632999 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 24701
I20260808 17:04:36.633036 25376 tablet_replica.cc:330] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: stopping tablet replica
I20260808 17:04:36.640012 25377 tablet_replica.cc:330] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: stopping tablet replica
I20260808 17:04:36.640184 25377 raft_consensus.cc:2229] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:36.640177 25376 raft_consensus.cc:2229] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:36.640288 25376 raft_consensus.cc:2258] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:36.640290 25377 raft_consensus.cc:2258] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:36.640741 25377 ts_tablet_manager.cc:1905] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:36.640750 25376 ts_tablet_manager.cc:1905] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:36.642203 25377 ts_tablet_manager.cc:1918] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:36.642225 25376 ts_tablet_manager.cc:1918] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:36.642336 25377 log.cc:1196] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/d7d87d75006c400e975a8a90ebceeec2
I20260808 17:04:36.642354 25376 log.cc:1196] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/d7d87d75006c400e975a8a90ebceeec2
I20260808 17:04:36.642642 25377 ts_tablet_manager.cc:1939] T d7d87d75006c400e975a8a90ebceeec2 P 7766a88f15b3404a911286aca42da04c: Deleting consensus metadata
I20260808 17:04:36.642726 25376 ts_tablet_manager.cc:1939] T d7d87d75006c400e975a8a90ebceeec2 P cc9ba4a566624e7c8f32616bf133b301: Deleting consensus metadata
I20260808 17:04:36.643239 24579 catalog_manager.cc:4841] TS cc9ba4a566624e7c8f32616bf133b301 (127.30.194.2:39323): tablet d7d87d75006c400e975a8a90ebceeec2 (table test-table [id=0f40fb4a45ed4bb4a33b4b28f2e59de8]) successfully deleted
I20260808 17:04:36.643522 24579 catalog_manager.cc:4841] TS 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443): tablet d7d87d75006c400e975a8a90ebceeec2 (table test-table [id=0f40fb4a45ed4bb4a33b4b28f2e59de8]) successfully deleted
W20260808 17:04:36.649011 24579 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:41241 (error 108)
I20260808 17:04:36.649252 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 24918
I20260808 17:04:36.649416 25330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:36.649316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:41241 (local address 127.0.0.1:33048)
0808 17:04:36.649362 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:36.649391 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:41241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 17:04:36.652412 24579 catalog_manager.cc:4584] TS bb3686e4e6a845fbafa65fdc1b9be694 (127.30.194.1:41241): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d7d87d75006c400e975a8a90ebceeec2: Network error: Client connection negotiation failed: client connection to 127.30.194.1:41241: connect: Connection refused (error 111)
I20260808 17:04:36.657234 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 25123
I20260808 17:04:36.668953 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 24544
I20260808 17:04:56.681296 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43273
--webserver_interface=127.30.194.62
--webserver_port=37161
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:04:56.762512 25380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:56.762686 25380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:56.762705 25380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:56.763887 25380 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 17:04:56.763926 25380 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 17:04:56.763938 25380 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 17:04:56.763948 25380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:04:56.763972 25380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:56.763996 25380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:56.764019 25380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:04:56.764041 25380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:04:56.765496 25380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=37161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:56.765705 25380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:56.765902 25380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:56.774171 25391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:56.774171 25387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:56.774418 25380 server_base.cc:1034] running on GCE node
W20260808 17:04:56.774163 25388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:56.774736 25380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:56.775012 25380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:56.776149 25380 hybrid_clock.cc:648] HybridClock initialized: now 1786208696776140 us; error 26 us; skew 500 ppm
Aug 08 17:04:56 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208696, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:56.781890 25380 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:04:56.782271 25380 webserver.cc:460] Webserver started at http://127.30.194.62:37161/ using document root <none> and password file <none>
I20260808 17:04:56.782511 25380 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:56.782577 25380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:56.783840 25380 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:56.784441 25400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:56.784595 25380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:56.784678 25380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "d62ea5d867294f23aa04748e29320bea"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "b7a129aaeeaa81f9aae1a9af61ef34b7"
server_key_iv: "e1f4f91d9263c2dc0f98dc209ec23f30"
server_key_version: "encryptionkey@0"
I20260808 17:04:56.785003 25380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:56.793732 25380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:56.795356 25380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:56.795482 25380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:56.799190 25380 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:43273
I20260808 17:04:56.799252 25483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:43273 every 8 connection(s)
I20260808 17:04:56.799575 25380 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:04:56.800881 25484 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:04:56.801548 25484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Bootstrap starting.
I20260808 17:04:56.806708 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 25380
I20260808 17:04:56.806978 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:41241
--local_ip_for_outbound_sockets=127.30.194.1
--tserver_master_addrs=127.30.194.62:43273
--webserver_port=43569
--webserver_interface=127.30.194.1
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:56.809232 25484 log.cc:824] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:56.810930 25484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:04:56.811177 25484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Bootstrap complete.
I20260808 17:04:56.813189 25484 raft_consensus.cc:348] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:04:56.813412 25484 raft_consensus.cc:729] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ea5d867294f23aa04748e29320bea, State: Initialized, Role: FOLLOWER
I20260808 17:04:56.813561 25484 consensus_queue.cc:260] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:04:56.813632 25484 raft_consensus.cc:388] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:04:56.813685 25484 raft_consensus.cc:482] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:04:56.813731 25484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:04:56.814623 25484 raft_consensus.cc:504] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:04:56.814713 25484 leader_election.cc:302] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d62ea5d867294f23aa04748e29320bea; no voters:
I20260808 17:04:56.814857 25484 leader_election.cc:288] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 17:04:56.814904 25489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:04:56.815053 25489 raft_consensus.cc:686] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 2 LEADER]: Becoming Leader. State: Replica: d62ea5d867294f23aa04748e29320bea, State: Running, Role: LEADER
I20260808 17:04:56.815106 25484 sys_catalog.cc:564] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:04:56.815162 25489 consensus_queue.cc:237] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:04:56.815448 25489 sys_catalog.cc:455] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: SysCatalogTable state changed. Reason: New leader d62ea5d867294f23aa04748e29320bea. Latest consensus state: current_term: 2 leader_uuid: "d62ea5d867294f23aa04748e29320bea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } } }
I20260808 17:04:56.815672 25489 sys_catalog.cc:458] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:56.815608 25490 sys_catalog.cc:455] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d62ea5d867294f23aa04748e29320bea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } } }
I20260808 17:04:56.815876 25490 sys_catalog.cc:458] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: This master's current role is: LEADER
I20260808 17:04:56.816119 25500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:04:56.816895 25500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:04:56.816994 25500 catalog_manager.cc:1259] Loaded cluster ID: e25e63b3dc7e43c9bf8a5ccf24aeba4b
I20260808 17:04:56.817025 25500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:04:56.818154 25500 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:04:56.818341 25500 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Loaded TSK: 0
I20260808 17:04:56.827982 25500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Generated new TSK 1
I20260808 17:04:56.828061 25500 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:04:56.890075 25486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:56.890259 25486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:56.890290 25486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:56.891395 25486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:56.891433 25486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:56.891469 25486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:04:56.892947 25486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:41241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=43569
--tserver_master_addrs=127.30.194.62:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:56.893277 25486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:56.893491 25486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:56.898219 25522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:56.898228 25518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:56.898381 25486 server_base.cc:1034] running on GCE node
W20260808 17:04:56.901227 25519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:56.901441 25486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:56.901674 25486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:56.902817 25486 hybrid_clock.cc:648] HybridClock initialized: now 1786208696902802 us; error 31 us; skew 500 ppm
Aug 08 17:04:56 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208696, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:56.905284 25486 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:04:56.905714 25486 webserver.cc:460] Webserver started at http://127.30.194.1:43569/ using document root <none> and password file <none>
I20260808 17:04:56.905952 25486 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:56.906014 25486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:56.907253 25486 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:04:56.907848 25532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:56.908025 25486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:04:56.908123 25486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "bb3686e4e6a845fbafa65fdc1b9be694"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "8eb10382aaaf321b1c504c8f469a0ac2"
server_key_iv: "4ff4cbd7b0633545cb167c81cc86a239"
server_key_version: "encryptionkey@0"
I20260808 17:04:56.908511 25486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:56.921815 25486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:56.924000 25486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:56.924129 25486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:56.924355 25486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:56.924816 25545 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:04:56.926259 25486 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:04:56.926317 25486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 17:04:56.926349 25486 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:04:56.927168 25486 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:04:56.927212 25486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 17:04:56.927268 25545 tablet_bootstrap.cc:492] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Bootstrap starting.
I20260808 17:04:56.934051 25486 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:41241
I20260808 17:04:56.934244 25708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:41241 every 8 connection(s)
I20260808 17:04:56.934465 25486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:04:56.937328 25545 log.cc:824] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:56.939759 25545 tablet_bootstrap.cc:492] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:04:56.940053 25545 tablet_bootstrap.cc:492] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Bootstrap complete.
I20260808 17:04:56.941170 25545 ts_tablet_manager.cc:1397] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.009s
Aug 08 17:04:56 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208696, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:56.942322 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 25486
I20260808 17:04:56.942346 25545 raft_consensus.cc:348] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:56.942621 25545 raft_consensus.cc:729] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb3686e4e6a845fbafa65fdc1b9be694, State: Initialized, Role: FOLLOWER
I20260808 17:04:56.942636 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:39323
--local_ip_for_outbound_sockets=127.30.194.2
--tserver_master_addrs=127.30.194.62:43273
--webserver_port=32931
--webserver_interface=127.30.194.2
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:56.942744 25545 consensus_queue.cc:260] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } }
I20260808 17:04:56.943009 25545 ts_tablet_manager.cc:1428] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 17:04:56.945762 25712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:56.935108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:43097)
0808 17:04:56.935429 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:56.935484 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:56.936106 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:56.936808 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:56.936813 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:56.937154 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:56.937408 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:56.937413 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:56.938617 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:56.938618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:56.939421 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:56.939423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:56.940040 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:56.940041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:56.940173 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:56.940631 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:56.940639 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:56.941919 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:56.943921 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:56.943928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:56.943931 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:56.944249 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:56.944542 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:56.944543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:56.944544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:56.944672 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:56.945120 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:56.945128 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:56.945290 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 17:04:56.945416 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 17:04:56.945574 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":82,"threads_started":1}
I20260808 17:04:56.945791 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:56.935215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:43097 (local address 127.30.194.62:43273)
0808 17:04:56.936321 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:56.936334 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:56.936380 (+ 46us) server_negotiation.cc:407] Connection header received
0808 17:04:56.936532 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:56.936539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:56.936600 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:56.936766 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:56.937525 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:56.938487 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:56.939510 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:56.939937 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:56.940043 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:56.942017 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:56.942021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:56.942022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:56.942033 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:56.943828 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:56.944346 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:56.944349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:56.944350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:56.944465 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:56.944740 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:56.944741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:56.944742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:56.945008 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:56.945119 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:56.945483 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 17:04:56.945614 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1047,"thread_start_us":88,"threads_started":1}
I20260808 17:04:56.946964 25709 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43273
I20260808 17:04:56.947057 25709 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:56.947216 25709 heartbeater.cc:511] Master 127.30.194.62:43273 requested a full tablet report, sending...
I20260808 17:04:56.947873 25424 ts_manager.cc:194] Registered new tserver with Master: bb3686e4e6a845fbafa65fdc1b9be694 (127.30.194.1:41241)
I20260808 17:04:56.948987 25424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:43097
I20260808 17:04:56.949945 25709 heartbeater.cc:503] Master 127.30.194.62:43273 was elected leader, sending a full tablet report...
I20260808 17:04:56.955760 25727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:56.948708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36746 (local address 127.30.194.1:41241)
0808 17:04:56.952612 (+ 3904us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:56.952615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:56.952624 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:04:56.952649 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:56.952650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:56.952662 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:56.952734 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:56.952915 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:56.953565 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:56.955034 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:56.955455 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:56.955509 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:56.955542 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:56.955685 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 17:04:56.955710 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3877,"thread_start_us":62,"threads_started":1}
I20260808 17:04:56.955801 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:56.948472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:41241 (local address 127.0.0.1:36746)
0808 17:04:56.948852 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:56.948890 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:56.948954 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:56.952770 (+ 3816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:56.952773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:56.952776 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:56.952840 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:56.952842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:56.953636 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:56.953637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:56.954948 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:56.954951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:56.955522 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:56.955523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:56.955598 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:56.955611 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:04:56.955675 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:04:56.955727 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":79,"threads_started":1}
I20260808 17:04:56.956195 25617 tablet_service.cc:1514] Processing DeleteTablet for tablet d7d87d75006c400e975a8a90ebceeec2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:36 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:36746
I20260808 17:04:56.956604 25729 tablet_replica.cc:330] stopping tablet replica
I20260808 17:04:56.956693 25729 raft_consensus.cc:2229] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:56.956754 25729 raft_consensus.cc:2258] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:56.956997 25729 ts_tablet_manager.cc:1905] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:56.958907 25729 ts_tablet_manager.cc:1918] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:56.958971 25729 log.cc:1196] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal/wals/d7d87d75006c400e975a8a90ebceeec2
I20260808 17:04:56.959152 25729 ts_tablet_manager.cc:1939] T d7d87d75006c400e975a8a90ebceeec2 P bb3686e4e6a845fbafa65fdc1b9be694: Deleting consensus metadata
I20260808 17:04:56.959615 25408 catalog_manager.cc:4841] TS bb3686e4e6a845fbafa65fdc1b9be694 (127.30.194.1:41241): tablet d7d87d75006c400e975a8a90ebceeec2 (table test-table [id=0f40fb4a45ed4bb4a33b4b28f2e59de8]) successfully deleted
W20260808 17:04:57.028301 25722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:57.028478 25722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:57.028499 25722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:57.029592 25722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:57.029629 25722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:57.029654 25722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:04:57.031018 25722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:39323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=32931
--tserver_master_addrs=127.30.194.62:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:57.031188 25722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:57.031355 25722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:57.035933 25737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:57.036016 25722 server_base.cc:1034] running on GCE node
W20260808 17:04:57.035912 25740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:57.035948 25736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:57.036265 25722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:57.036463 25722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:57.037595 25722 hybrid_clock.cc:648] HybridClock initialized: now 1786208697037585 us; error 27 us; skew 500 ppm
Aug 08 17:04:57 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208697, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:57.039816 25722 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:04:57.040186 25722 webserver.cc:460] Webserver started at http://127.30.194.2:32931/ using document root <none> and password file <none>
I20260808 17:04:57.040388 25722 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:57.040452 25722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:57.041699 25722 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:57.042219 25748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:57.042357 25722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:04:57.042436 25722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "cc9ba4a566624e7c8f32616bf133b301"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "1d9a92b7a84d6ca154485536d0d6e6be"
server_key_iv: "1d1b6825efe0d95feada0a7bd13146c8"
server_key_version: "encryptionkey@0"
I20260808 17:04:57.042704 25722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:57.062640 25722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:57.064487 25722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:57.064615 25722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:57.064815 25722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:57.065169 25722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:57.065207 25722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:57.065235 25722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:57.065253 25722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:57.070583 25722 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:39323
I20260808 17:04:57.070626 25922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:39323 every 8 connection(s)
I20260808 17:04:57.070925 25722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:04:57 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208697, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:57.077848 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 25722
I20260808 17:04:57.078102 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:42443
--local_ip_for_outbound_sockets=127.30.194.3
--tserver_master_addrs=127.30.194.62:43273
--webserver_port=34753
--webserver_interface=127.30.194.3
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:04:57.079341 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.071565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:49917 (local address 127.30.194.62:43273)
0808 17:04:57.071621 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:57.071624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:57.071805 (+ 181us) server_negotiation.cc:407] Connection header received
0808 17:04:57.072299 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:57.072300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:57.072319 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:57.072372 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:57.073285 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.073767 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.074721 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.074925 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.074981 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.076876 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:57.076878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:57.076879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:57.076886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:57.078062 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:57.078458 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:57.078459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:57.078460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:57.078515 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:57.078729 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:57.078731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:57.078731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:57.078845 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:57.078902 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:57.079148 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 17:04:57.079238 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:04:57.079476 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.071535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:49917)
0808 17:04:57.071683 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:57.071724 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:57.072197 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:57.072443 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:57.072448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:57.072985 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:57.073184 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.073189 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.073849 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.073850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.074632 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.074635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.074994 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.074994 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.075120 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.075535 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:57.075543 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:57.076785 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:57.078125 (+ 1340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:57.078133 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:57.078135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:57.078389 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:57.078568 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:57.078569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:57.078570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:57.078673 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:57.078901 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:57.078907 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:57.079034 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 17:04:57.079120 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 17:04:57.079264 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260808 17:04:57.079864 25925 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43273
I20260808 17:04:57.079952 25925 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:57.080112 25925 heartbeater.cc:511] Master 127.30.194.62:43273 requested a full tablet report, sending...
I20260808 17:04:57.080476 25424 ts_manager.cc:194] Registered new tserver with Master: cc9ba4a566624e7c8f32616bf133b301 (127.30.194.2:39323)
I20260808 17:04:57.080826 25424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:49917
W20260808 17:04:57.158860 25929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:04:57.159032 25929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:04:57.159054 25929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:04:57.160166 25929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:04:57.160202 25929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:04:57.160231 25929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:04:57.161631 25929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:42443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=34753
--tserver_master_addrs=127.30.194.62:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:04:57.161836 25929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:04:57.162027 25929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:04:57.166626 25938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:57.166641 25935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:04:57.166649 25934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:04:57.166829 25929 server_base.cc:1034] running on GCE node
I20260808 17:04:57.167030 25929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:04:57.167236 25929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:04:57.168380 25929 hybrid_clock.cc:648] HybridClock initialized: now 1786208697168369 us; error 27 us; skew 500 ppm
Aug 08 17:04:57 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208697, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:04:57.170696 25929 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:04:57.171083 25929 webserver.cc:460] Webserver started at http://127.30.194.3:34753/ using document root <none> and password file <none>
I20260808 17:04:57.171285 25929 fs_manager.cc:359] Metadata directory not provided
I20260808 17:04:57.171350 25929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:04:57.172545 25929 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:04:57.173077 25948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:57.173228 25929 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:57.173308 25929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "7766a88f15b3404a911286aca42da04c"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "e76b0d8aa22c4d2cb1aaf7c12c58b728"
server_key_iv: "134339300bf60c965c0eeb7f346b33a8"
server_key_version: "encryptionkey@0"
I20260808 17:04:57.173573 25929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:04:57.199148 25929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:04:57.200845 25929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:04:57.200970 25929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:04:57.201184 25929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:04:57.201465 25929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:04:57.201498 25929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:57.201526 25929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:04:57.201542 25929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:57.207087 25929 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:42443
I20260808 17:04:57.207124 26126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:42443 every 8 connection(s)
I20260808 17:04:57.207446 25929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:04:57 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208697, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:04:57.213845 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 25929
I20260808 17:04:57.216111 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.208045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:52977 (local address 127.30.194.62:43273)
0808 17:04:57.208085 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:57.208087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:57.208282 (+ 195us) server_negotiation.cc:407] Connection header received
0808 17:04:57.208765 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:57.208767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:57.208779 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:57.208842 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:57.209775 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.210264 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.211203 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.211402 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.211458 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.213447 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:57.213448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:57.213449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:57.213456 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:57.214779 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:57.215225 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:57.215227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:57.215227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:57.215292 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:57.215511 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:57.215512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:57.215512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:57.215618 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:57.215674 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:57.215941 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 17:04:57.216022 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:04:57.216351 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.208034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:52977)
0808 17:04:57.208173 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:57.208207 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:57.208671 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:57.208914 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:57.208918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:57.209501 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:04:57.209679 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.209684 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.210349 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.210350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.211117 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.211120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.211470 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.211471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.211592 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.211976 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:04:57.211983 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:04:57.213343 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:04:57.214860 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:57.214867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:57.214869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:57.215091 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:57.215355 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:04:57.215356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:04:57.215356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:04:57.215454 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:04:57.215671 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:04:57.215678 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:04:57.215820 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 17:04:57.215920 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 17:04:57.216069 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260808 17:04:57.216766 26127 heartbeater.cc:347] Connected to a master server at 127.30.194.62:43273
I20260808 17:04:57.216864 26127 heartbeater.cc:464] Registering TS with master...
I20260808 17:04:57.217047 26127 heartbeater.cc:511] Master 127.30.194.62:43273 requested a full tablet report, sending...
I20260808 17:04:57.217492 25424 ts_manager.cc:194] Registered new tserver with Master: 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443)
I20260808 17:04:57.217967 25424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:52977
I20260808 17:04:57.220460 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.214231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.30.194.62:43273)
0808 17:04:57.214330 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:57.214332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:57.214470 (+ 138us) server_negotiation.cc:407] Connection header received
0808 17:04:57.214530 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:57.214531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:57.214543 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:57.214598 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:57.215145 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.215627 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.216601 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.216876 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.216938 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.218297 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:57.218298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:57.218299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:57.218305 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:57.219515 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:57.219807 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:57.219808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:57.219808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:57.219862 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:57.220017 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:57.220018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:57.220019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:57.220119 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:57.220269 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:57.220303 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 17:04:57.220376 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":53,"threads_started":1}
I20260808 17:04:57.221149 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:04:57.223397 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.221475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45576 (local address 127.30.194.62:43273)
0808 17:04:57.221541 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:57.221542 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:57.221554 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:04:57.221611 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:57.221612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:57.221636 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:57.221712 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:04:57.221779 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.222200 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.222871 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.223053 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.223107 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.223207 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:04:57.223243 (+ 36us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 17:04:57.223359 (+ 116us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45576: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 17:04:57.223438 26130 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45576: token expired
W20260808 17:04:57.223572 25323 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.194.62:43273)
I20260808 17:04:57.228493 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.223795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.30.194.62:43273)
0808 17:04:57.223851 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:57.223853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:57.223860 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:04:57.223876 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:57.223877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:57.223884 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:57.223926 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:04:57.224336 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.224757 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.225457 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.225628 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.225677 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.226598 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:04:57.226599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:04:57.226599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:04:57.226606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:04:57.227546 (+ 940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:57.227830 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:57.227831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:57.227831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:57.227878 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:04:57.228034 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:04:57.228035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:04:57.228035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:04:57.228152 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:04:57.228233 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:57.228352 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 17:04:57.228417 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:04:57.229560 25424 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:04:57.231045 25424 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:04:57.233443 25617 tablet_service.cc:1467] Processing CreateTablet for tablet ffb475cd195946b2a9e95db8ec08aa21 (DEFAULT_TABLE table=test-table [id=7eb26bfeddd94c3aac194aabc2f0914e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:57.233639 25617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb475cd195946b2a9e95db8ec08aa21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:57.235057 25545 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Bootstrap starting.
I20260808 17:04:57.235388 25545 tablet_bootstrap.cc:654] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:57.236032 25545 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: No bootstrap required, opened a new log
I20260808 17:04:57.236089 25545 ts_tablet_manager.cc:1397] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 17:04:57.236189 25545 raft_consensus.cc:348] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:04:57.236259 25545 raft_consensus.cc:374] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:57.236323 25545 raft_consensus.cc:729] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb3686e4e6a845fbafa65fdc1b9be694, State: Initialized, Role: FOLLOWER
I20260808 17:04:57.236371 25545 consensus_queue.cc:260] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:04:57.236460 25545 ts_tablet_manager.cc:1428] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:04:57.236639 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.232836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:39323 (local address 127.0.0.1:39638)
0808 17:04:57.232905 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:57.232915 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:57.232955 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:57.233340 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:57.233341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:57.233343 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:57.233382 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.233384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.234547 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.234548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.235572 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.235576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.236393 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.236395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.236458 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.236462 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:57.236527 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 17:04:57.236568 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 17:04:57.236747 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.232916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39638 (local address 127.30.194.2:39323)
0808 17:04:57.233177 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:57.233181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:57.233192 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:04:57.233227 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:57.233228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:57.233240 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:57.233328 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:57.233460 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.234453 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.235661 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.236320 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.236392 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.236452 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:57.236627 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 17:04:57.236667 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":93,"threads_started":1}
I20260808 17:04:57.237461 25827 tablet_service.cc:1467] Processing CreateTablet for tablet ffb475cd195946b2a9e95db8ec08aa21 (DEFAULT_TABLE table=test-table [id=7eb26bfeddd94c3aac194aabc2f0914e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:57.237830 25827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb475cd195946b2a9e95db8ec08aa21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:57.238029 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.232940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:42443 (local address 127.0.0.1:50794)
0808 17:04:57.233124 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:57.233134 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:57.233177 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:57.234560 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:57.234561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:57.234564 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:57.234603 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.234604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.236007 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.236008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.237011 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.237014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.237781 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.237782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.237857 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.237861 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:57.237923 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 17:04:57.237958 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260808 17:04:57.238045 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.233011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50794 (local address 127.30.194.3:42443)
0808 17:04:57.233333 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:57.233337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:57.233346 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:04:57.233383 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:57.233385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:57.233401 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:57.233482 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:57.234680 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.235924 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.237130 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.237695 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.237777 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.237832 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:57.237937 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 17:04:57.237975 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":104,"threads_started":1}
I20260808 17:04:57.238646 26026 tablet_service.cc:1467] Processing CreateTablet for tablet ffb475cd195946b2a9e95db8ec08aa21 (DEFAULT_TABLE table=test-table [id=7eb26bfeddd94c3aac194aabc2f0914e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:04:57.238933 26026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb475cd195946b2a9e95db8ec08aa21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:04:57.240636 26137 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Bootstrap starting.
I20260808 17:04:57.241348 26137 tablet_bootstrap.cc:654] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:57.241969 26137 log.cc:824] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:57.242027 26139 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Bootstrap starting.
I20260808 17:04:57.242544 26139 tablet_bootstrap.cc:654] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Neither blocks nor log segments found. Creating new log.
I20260808 17:04:57.242796 26137 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: No bootstrap required, opened a new log
I20260808 17:04:57.242878 26137 ts_tablet_manager.cc:1397] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:04:57.243054 26139 log.cc:824] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Log is configured to *not* fsync() on all Append() calls
I20260808 17:04:57.243665 26139 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: No bootstrap required, opened a new log
I20260808 17:04:57.243723 26139 ts_tablet_manager.cc:1397] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:04:57.244318 26137 raft_consensus.cc:348] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:04:57.244463 26137 raft_consensus.cc:374] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:57.244486 26137 raft_consensus.cc:729] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9ba4a566624e7c8f32616bf133b301, State: Initialized, Role: FOLLOWER
I20260808 17:04:57.244603 26137 consensus_queue.cc:260] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:04:57.244887 25925 heartbeater.cc:503] Master 127.30.194.62:43273 was elected leader, sending a full tablet report...
I20260808 17:04:57.244870 26137 ts_tablet_manager.cc:1428] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:04:57.245257 26139 raft_consensus.cc:348] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:04:57.245388 26139 raft_consensus.cc:374] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:04:57.245409 26139 raft_consensus.cc:729] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7766a88f15b3404a911286aca42da04c, State: Initialized, Role: FOLLOWER
I20260808 17:04:57.245541 26139 consensus_queue.cc:260] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:04:57.245810 26139 ts_tablet_manager.cc:1428] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:04:57.245819 26127 heartbeater.cc:503] Master 127.30.194.62:43273 was elected leader, sending a full tablet report...
W20260808 17:04:57.321420 25926 tablet.cc:2369] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:04:57.410032 25721 raft_consensus.cc:482] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:04:57.410226 25721 raft_consensus.cc:504] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:04:57.410612 25721 leader_election.cc:288] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc9ba4a566624e7c8f32616bf133b301 (127.30.194.2:39323), 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443)
I20260808 17:04:57.413378 25712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.410810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:39323 (local address 127.30.194.1:42111)
0808 17:04:57.410869 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:57.410881 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:57.410932 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:57.411111 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:57.411113 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:57.411115 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:57.411178 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.411180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.411773 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.411774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.412544 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.412547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.413121 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.413122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.413190 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.413194 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:57.413241 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:04:57.413277 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:04:57.413424 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.410899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:42111 (local address 127.30.194.2:39323)
0808 17:04:57.410967 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:57.410969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:57.410980 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:04:57.411012 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:57.411013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:57.411022 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:57.411087 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:57.411237 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.411700 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.412632 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.413039 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.413123 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.413133 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:57.413325 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 17:04:57.413366 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 17:04:57.413807 25858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb475cd195946b2a9e95db8ec08aa21" candidate_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9ba4a566624e7c8f32616bf133b301" is_pre_election: true
I20260808 17:04:57.413992 25858 raft_consensus.cc:2454] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb3686e4e6a845fbafa65fdc1b9be694 in term 0.
I20260808 17:04:57.414096 26146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.410929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:42443 (local address 127.30.194.1:44933)
0808 17:04:57.411026 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:57.411034 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:04:57.411076 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:04:57.411213 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:04:57.411214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:04:57.411215 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:04:57.411529 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.411531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.412318 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.412319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.413109 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:04:57.413112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.413843 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.413844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:04:57.413898 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.413902 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:04:57.413964 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 17:04:57.414002 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":70,"threads_started":1}
I20260808 17:04:57.414125 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.411016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:44933 (local address 127.30.194.3:42443)
0808 17:04:57.411077 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:57.411080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:57.411107 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:04:57.411137 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:57.411138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:57.411148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:57.411198 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:04:57.411607 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.412243 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.413234 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.413717 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.413794 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.413803 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:57.414023 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 17:04:57.414062 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:04:57.414312 25538 leader_election.cc:302] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb3686e4e6a845fbafa65fdc1b9be694, cc9ba4a566624e7c8f32616bf133b301; no voters:
I20260808 17:04:57.414444 26054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb475cd195946b2a9e95db8ec08aa21" candidate_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7766a88f15b3404a911286aca42da04c" is_pre_election: true
I20260808 17:04:57.414453 25721 raft_consensus.cc:2781] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:04:57.414543 25721 raft_consensus.cc:482] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:04:57.414566 25721 raft_consensus.cc:3037] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:57.414597 26054 raft_consensus.cc:2454] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb3686e4e6a845fbafa65fdc1b9be694 in term 0.
I20260808 17:04:57.415364 25721 raft_consensus.cc:504] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:04:57.415472 25721 leader_election.cc:288] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 1 election: Requested vote from peers cc9ba4a566624e7c8f32616bf133b301 (127.30.194.2:39323), 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443)
I20260808 17:04:57.415711 26054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb475cd195946b2a9e95db8ec08aa21" candidate_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7766a88f15b3404a911286aca42da04c"
I20260808 17:04:57.415719 25858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb475cd195946b2a9e95db8ec08aa21" candidate_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9ba4a566624e7c8f32616bf133b301"
I20260808 17:04:57.415791 25858 raft_consensus.cc:3037] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:57.415789 26054 raft_consensus.cc:3037] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:04:57.416507 26054 raft_consensus.cc:2454] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb3686e4e6a845fbafa65fdc1b9be694 in term 1.
I20260808 17:04:57.416613 25858 raft_consensus.cc:2454] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb3686e4e6a845fbafa65fdc1b9be694 in term 1.
I20260808 17:04:57.416677 25538 leader_election.cc:302] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7766a88f15b3404a911286aca42da04c, bb3686e4e6a845fbafa65fdc1b9be694; no voters:
I20260808 17:04:57.416793 25721 raft_consensus.cc:2781] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:04:57.416967 25721 raft_consensus.cc:686] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 LEADER]: Becoming Leader. State: Replica: bb3686e4e6a845fbafa65fdc1b9be694, State: Running, Role: LEADER
I20260808 17:04:57.417035 25721 consensus_queue.cc:237] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:04:57.417851 25424 catalog_manager.cc:5495] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb3686e4e6a845fbafa65fdc1b9be694 (127.30.194.1). New cstate: current_term: 1 leader_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } health_report { overall_health: HEALTHY } } }
W20260808 17:04:57.435348 25711 tablet.cc:2369] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:04:57.457938 26128 tablet.cc:2369] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:04:57.462208 26151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.460113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36762 (local address 127.30.194.1:41241)
0808 17:04:57.460234 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 17:04:57.460236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:04:57.460244 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:04:57.460274 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:04:57.460275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:04:57.460293 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:04:57.460367 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:04:57.460464 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.460920 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.461636 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:04:57.461824 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:04:57.461866 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:04:57.461959 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:04:57.462055 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:04:57.462103 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:04:57.462130 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 17:04:57.462146 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":50,"threads_started":1}
I20260808 17:04:57.463603 26054 raft_consensus.cc:1261] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 1 FOLLOWER]: Refusing update from remote peer bb3686e4e6a845fbafa65fdc1b9be694: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:57.463620 25858 raft_consensus.cc:1261] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Refusing update from remote peer bb3686e4e6a845fbafa65fdc1b9be694: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:04:57.463913 25721 consensus_queue.cc:1035] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:57.464037 26149 consensus_queue.cc:1035] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:04:57.467171 26155 mvcc.cc:205] Tried to move back new op lower bound from 7316310824808501248 to 7316310824620883968. Current Snapshot: MvccSnapshot[applied={T|T < 7316310824808501248}]
I20260808 17:04:57.467279 26156 mvcc.cc:205] Tried to move back new op lower bound from 7316310824808501248 to 7316310824620883968. Current Snapshot: MvccSnapshot[applied={T|T < 7316310824808501248}]
I20260808 17:04:57.468385 25423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:04:57.468470 25423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:04:57.469133 25423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Sending DeleteTablet for 3 replicas of tablet ffb475cd195946b2a9e95db8ec08aa21
I20260808 17:04:57.469412 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 25486
I20260808 17:04:57.469427 25617 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb475cd195946b2a9e95db8ec08aa21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:57 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:36746
I20260808 17:04:57.469436 26026 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb475cd195946b2a9e95db8ec08aa21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:57 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:50794
I20260808 17:04:57.469518 25827 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb475cd195946b2a9e95db8ec08aa21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:04:57 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:39638
W20260808 17:04:57.484937 25408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:41241 (error 108)
I20260808 17:04:57.485073 26170 tablet_replica.cc:330] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: stopping tablet replica
I20260808 17:04:57.485150 26169 tablet_replica.cc:330] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: stopping tablet replica
I20260808 17:04:57.485194 26170 raft_consensus.cc:2229] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:57.485278 26169 raft_consensus.cc:2229] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:04:57.485323 26170 raft_consensus.cc:2258] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:57.485463 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 25722
I20260808 17:04:57.485569 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.485446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:41241 (local address 127.0.0.1:36764)
0808 17:04:57.485495 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:57.485535 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:41241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 17:04:57.492143 25408 catalog_manager.cc:4584] TS bb3686e4e6a845fbafa65fdc1b9be694 (127.30.194.1:41241): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ffb475cd195946b2a9e95db8ec08aa21: Network error: Client connection negotiation failed: client connection to 127.30.194.1:41241: connect: Connection refused (error 111)
I20260808 17:04:57.485447 26169 raft_consensus.cc:2258] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:04:57.494220 26169 ts_tablet_manager.cc:1905] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:04:57.496166 26169 ts_tablet_manager.cc:1918] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:04:57.496271 26169 log.cc:1196] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/ffb475cd195946b2a9e95db8ec08aa21
I20260808 17:04:57.496546 26169 ts_tablet_manager.cc:1939] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Deleting consensus metadata
I20260808 17:04:57.497387 25408 catalog_manager.cc:4841] TS 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443): tablet ffb475cd195946b2a9e95db8ec08aa21 (table test-table [id=7eb26bfeddd94c3aac194aabc2f0914e]) successfully deleted
I20260808 17:04:57.500737 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:04:57.500610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:39323 (local address 127.0.0.1:39652)
0808 17:04:57.500665 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 17:04:57.500702 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.2:39323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:04:57.500739 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 25929
I20260808 17:04:57.508764 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 25380
I20260808 17:05:17.521965 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43273
--webserver_interface=127.30.194.62
--webserver_port=37161
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:17.603410 26173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:17.603595 26173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:17.603613 26173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:17.604677 26173 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 17:05:17.604707 26173 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 17:05:17.604718 26173 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 17:05:17.604728 26173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:17.604739 26173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:17.604751 26173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:17.604761 26173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:17.604771 26173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:17.606256 26173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=37161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:17.606455 26173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:17.606617 26173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:17.611357 26180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:17.611392 26181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:17.611392 26186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:17.611428 26173 server_base.cc:1034] running on GCE node
I20260808 17:05:17.611794 26173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:17.612052 26173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:17.613183 26173 hybrid_clock.cc:648] HybridClock initialized: now 1786208717613172 us; error 26 us; skew 500 ppm
Aug 08 17:05:17 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:17.619956 26173 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:05:17.620397 26173 webserver.cc:460] Webserver started at http://127.30.194.62:37161/ using document root <none> and password file <none>
I20260808 17:05:17.620607 26173 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:17.620656 26173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:17.621917 26173 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:05:17.622524 26195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:17.622663 26173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:17.622746 26173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "d62ea5d867294f23aa04748e29320bea"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "b7a129aaeeaa81f9aae1a9af61ef34b7"
server_key_iv: "e1f4f91d9263c2dc0f98dc209ec23f30"
server_key_version: "encryptionkey@0"
I20260808 17:05:17.623034 26173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:17.636540 26173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:17.638182 26173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:17.638315 26173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:17.642037 26173 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:43273
I20260808 17:05:17.642066 26271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:43273 every 8 connection(s)
I20260808 17:05:17.642402 26173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:17.643694 26273 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:05:17.644261 26273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Bootstrap starting.
I20260808 17:05:17.647593 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 26173
I20260808 17:05:17.647855 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:41241
--local_ip_for_outbound_sockets=127.30.194.1
--tserver_master_addrs=127.30.194.62:43273
--webserver_port=43569
--webserver_interface=127.30.194.1
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:05:17.652047 26273 log.cc:824] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:17.654156 26273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:05:17.654415 26273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Bootstrap complete.
I20260808 17:05:17.656319 26273 raft_consensus.cc:348] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:05:17.656523 26273 raft_consensus.cc:729] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ea5d867294f23aa04748e29320bea, State: Initialized, Role: FOLLOWER
I20260808 17:05:17.656664 26273 consensus_queue.cc:260] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:05:17.656739 26273 raft_consensus.cc:388] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:17.656795 26273 raft_consensus.cc:482] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:17.656842 26273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 2 FOLLOWER]: Advancing to term 3
I20260808 17:05:17.657811 26273 raft_consensus.cc:504] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:05:17.657902 26273 leader_election.cc:302] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d62ea5d867294f23aa04748e29320bea; no voters:
I20260808 17:05:17.658044 26273 leader_election.cc:288] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 17:05:17.658109 26278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 3 FOLLOWER]: Leader election won for term 3
I20260808 17:05:17.658275 26273 sys_catalog.cc:564] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:17.658345 26278 raft_consensus.cc:686] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [term 3 LEADER]: Becoming Leader. State: Replica: d62ea5d867294f23aa04748e29320bea, State: Running, Role: LEADER
I20260808 17:05:17.658499 26278 consensus_queue.cc:237] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } }
I20260808 17:05:17.658860 26279 sys_catalog.cc:455] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d62ea5d867294f23aa04748e29320bea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } } }
I20260808 17:05:17.658944 26279 sys_catalog.cc:458] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:17.659161 26280 sys_catalog.cc:455] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: SysCatalogTable state changed. Reason: New leader d62ea5d867294f23aa04748e29320bea. Latest consensus state: current_term: 3 leader_uuid: "d62ea5d867294f23aa04748e29320bea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d62ea5d867294f23aa04748e29320bea" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43273 } } }
I20260808 17:05:17.659250 26280 sys_catalog.cc:458] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:17.659237 26288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:17.660156 26288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:17.660259 26288 catalog_manager.cc:1259] Loaded cluster ID: e25e63b3dc7e43c9bf8a5ccf24aeba4b
I20260808 17:05:17.660300 26288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:17.661432 26288 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:17.661605 26288 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Loaded TSK: 0
I20260808 17:05:17.661680 26288 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d62ea5d867294f23aa04748e29320bea: Loaded TSK: 1
F20260808 17:05:17.661908 26288 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f63383e99ad google::LogMessage::Fail() at ??:0
@ 0x7f63383ec88a google::LogMessage::SendToLog() at ??:0
@ 0x7f63383e94de google::LogMessage::Flush() at ??:0
@ 0x7f63383eaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f633ab56104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f633ab53d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f63403431cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f634034348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6340343594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6340309c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6340313ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f633a2b70d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f633a2ac954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f633b1106db start_thread at ??:0
@ 0x7f6336ee761f clone at ??:0
W20260808 17:05:17.747078 26276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:17.747264 26276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:17.747287 26276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:17.748370 26276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:17.748406 26276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:17.748430 26276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:05:17.749832 26276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:41241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=43569
--tserver_master_addrs=127.30.194.62:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:17.750027 26276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:17.750233 26276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:17.755923 26339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:17.755944 26336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:17.755930 26342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:17.756379 26276 server_base.cc:1034] running on GCE node
I20260808 17:05:17.756587 26276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:17.756868 26276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:17.758037 26276 hybrid_clock.cc:648] HybridClock initialized: now 1786208717758017 us; error 32 us; skew 500 ppm
Aug 08 17:05:17 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:17.760906 26276 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:05:17.761456 26276 webserver.cc:460] Webserver started at http://127.30.194.1:43569/ using document root <none> and password file <none>
I20260808 17:05:17.761696 26276 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:17.761776 26276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:17.763403 26276 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:05:17.764168 26354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:17.764343 26276 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:17.764439 26276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "bb3686e4e6a845fbafa65fdc1b9be694"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "8eb10382aaaf321b1c504c8f469a0ac2"
server_key_iv: "4ff4cbd7b0633545cb167c81cc86a239"
server_key_version: "encryptionkey@0"
I20260808 17:05:17.764868 26276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:17.779996 26276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:17.782208 26276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:17.782346 26276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:17.782575 26276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:17.783046 26363 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:05:17.784384 26276 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:05:17.784442 26276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 17:05:17.784473 26276 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:05:17.785351 26276 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:05:17.785394 26276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 17:05:17.785445 26363 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Bootstrap starting.
I20260808 17:05:17.792248 26276 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:41241
I20260808 17:05:17.792339 26533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:41241 every 8 connection(s)
I20260808 17:05:17.792761 26276 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:05:17.793262 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 26276
I20260808 17:05:17.793566 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:39323
--local_ip_for_outbound_sockets=127.30.194.2
--tserver_master_addrs=127.30.194.62:43273
--webserver_port=32931
--webserver_interface=127.30.194.2
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:05:17.795765 26363 log.cc:824] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:17.797858 26363 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:05:17.798121 26363 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Bootstrap complete.
I20260808 17:05:17.799095 26363 ts_tablet_manager.cc:1397] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260808 17:05:17.800426 26363 raft_consensus.cc:348] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:05:17.800784 26363 raft_consensus.cc:729] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb3686e4e6a845fbafa65fdc1b9be694, State: Initialized, Role: FOLLOWER
I20260808 17:05:17.800961 26363 consensus_queue.cc:260] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:05:17.801265 26363 ts_tablet_manager.cc:1428] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260808 17:05:17.855377 26538 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:05:17.793454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:52599)
0808 17:05:17.793791 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:17.793841 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:17.794441 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:17.855212 (+ 60771us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
W20260808 17:05:17.855628 26535 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:43273 (0 consecutive failures): Network error: Failed to ping master at 127.30.194.62:43273: Client connection negotiation failed: client connection to 127.30.194.62:43273: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 17:05:17.855949 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:17.855842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:58891)
0808 17:05:17.855891 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:17.855928 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 17:05:17.856264 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:17.856183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:44115)
0808 17:05:17.856220 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:17.856248 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 17:05:17.856371 26535 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:05:17.878705 26539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:17.878888 26539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:17.878909 26539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:17.879988 26539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:17.880025 26539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:17.880049 26539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:05:17.881453 26539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:39323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=32931
--tserver_master_addrs=127.30.194.62:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:17.881634 26539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:17.881803 26539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:17.886296 26546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:17.886399 26539 server_base.cc:1034] running on GCE node
W20260808 17:05:17.886297 26551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:17.886327 26547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:17.886713 26539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:17.886930 26539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:17.888073 26539 hybrid_clock.cc:648] HybridClock initialized: now 1786208717888047 us; error 31 us; skew 500 ppm
Aug 08 17:05:17 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:17.890419 26539 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:05:17.890789 26539 webserver.cc:460] Webserver started at http://127.30.194.2:32931/ using document root <none> and password file <none>
I20260808 17:05:17.891007 26539 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:17.891055 26539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:17.892226 26539 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:05:17.892824 26564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:17.892982 26539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:17.893060 26539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "cc9ba4a566624e7c8f32616bf133b301"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "1d9a92b7a84d6ca154485536d0d6e6be"
server_key_iv: "1d1b6825efe0d95feada0a7bd13146c8"
server_key_version: "encryptionkey@0"
I20260808 17:05:17.893378 26539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:17.917943 26539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:17.919613 26539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:17.919736 26539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:17.919934 26539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:17.920292 26577 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:05:17.921634 26539 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:05:17.921674 26539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260808 17:05:17.921703 26539 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:05:17.922283 26539 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:05:17.922314 26539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 17:05:17.922353 26577 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Bootstrap starting.
I20260808 17:05:17.928287 26539 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:39323
I20260808 17:05:17.928378 26746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:39323 every 8 connection(s)
I20260808 17:05:17.928675 26539 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:05:17.929728 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:17.929247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:53423)
0808 17:05:17.929438 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:17.929537 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
W20260808 17:05:17.929939 26748 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:43273 (0 consecutive failures): Network error: Failed to ping master at 127.30.194.62:43273: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
I20260808 17:05:17.930225 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:17.930115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:56541)
0808 17:05:17.930166 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:17.930195 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 17:05:17.930485 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:17.930407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:57951)
0808 17:05:17.930437 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:17.930466 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260808 17:05:17.930583 26748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:05:17.931448 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 26539
I20260808 17:05:17.931613 26577 log.cc:824] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:17.931720 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:42443
--local_ip_for_outbound_sockets=127.30.194.3
--tserver_master_addrs=127.30.194.62:43273
--webserver_port=34753
--webserver_interface=127.30.194.3
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:05:17.933679 26577 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:05:17.933967 26577 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Bootstrap complete.
I20260808 17:05:17.934878 26577 ts_tablet_manager.cc:1397] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260808 17:05:17.935981 26577 raft_consensus.cc:348] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:05:17.936209 26577 raft_consensus.cc:729] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9ba4a566624e7c8f32616bf133b301, State: Initialized, Role: FOLLOWER
I20260808 17:05:17.936312 26577 consensus_queue.cc:260] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:05:17.936535 26577 ts_tablet_manager.cc:1428] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:05:17.936765 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:17.936672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:34787)
0808 17:05:17.936713 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:17.936744 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 17:05:17.937006 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:17.936932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:57803)
0808 17:05:17.936967 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:17.936987 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 17:05:18.014433 26756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:18.014613 26756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:18.014644 26756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:18.015729 26756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:18.015767 26756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:18.015800 26756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:05:18.017263 26756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:42443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=34753
--tserver_master_addrs=127.30.194.62:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:18.017472 26756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:18.017688 26756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:18.022367 26764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:18.022500 26756 server_base.cc:1034] running on GCE node
W20260808 17:05:18.022357 26763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:18.022377 26770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:18.022806 26756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:18.023016 26756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:18.024163 26756 hybrid_clock.cc:648] HybridClock initialized: now 1786208718024149 us; error 27 us; skew 500 ppm
Aug 08 17:05:18 dist-test-slave-2j7r krb5kdc[24523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:18.026608 26756 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:05:18.026991 26756 webserver.cc:460] Webserver started at http://127.30.194.3:34753/ using document root <none> and password file <none>
I20260808 17:05:18.027192 26756 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:18.027237 26756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:18.028422 26756 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:05:18.028949 26779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:18.029114 26756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:18.029196 26756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "7766a88f15b3404a911286aca42da04c"
format_stamp: "Formatted at 2026-08-08 17:04:36 on dist-test-slave-2j7r"
server_key: "e76b0d8aa22c4d2cb1aaf7c12c58b728"
server_key_iv: "134339300bf60c965c0eeb7f346b33a8"
server_key_version: "encryptionkey@0"
I20260808 17:05:18.029485 26756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:18.049255 26756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:18.050791 26756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:18.050892 26756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:18.051052 26756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:18.051285 26756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:18.051316 26756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:18.051333 26756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:18.051349 26756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:18.056581 26756 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:42443
I20260808 17:05:18.056634 26952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:42443 every 8 connection(s)
I20260808 17:05:18.056949 26756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:05:18.057874 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:18.057525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:42555)
0808 17:05:18.057669 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:18.057744 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
W20260808 17:05:18.058243 26953 heartbeater.cc:644] Failed to heartbeat to 127.30.194.62:43273 (0 consecutive failures): Network error: Failed to ping master at 127.30.194.62:43273: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
I20260808 17:05:18.058472 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:18.058394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:45087)
0808 17:05:18.058430 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:18.058455 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 17:05:18.058682 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:18.058626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:32949)
0808 17:05:18.058654 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:18.058671 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 17:05:18.058766 26953 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:05:18.059365 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 26756
I20260808 17:05:18.856870 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:18.856664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:59055)
0808 17:05:18.856773 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:18.856811 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260808 17:05:18.937505 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:18.937295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:42699)
0808 17:05:18.937429 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:18.937463 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":56,"threads_started":1}
I20260808 17:05:19.059253 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.059055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:46351)
0808 17:05:19.059171 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:19.059202 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260808 17:05:19.408700 26966 raft_consensus.cc:482] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:05:19.408818 26966 raft_consensus.cc:504] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:05:19.409111 26966 leader_election.cc:288] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443), bb3686e4e6a845fbafa65fdc1b9be694 (127.30.194.1:41241)
Aug 08 17:05:19 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.1@KRBTEST.COM
Aug 08 17:05:19 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.3@KRBTEST.COM
I20260808 17:05:19.419889 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.409440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:56453 (local address 127.30.194.1:41241)
0808 17:05:19.409601 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:19.409605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:19.409613 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:19.409691 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:19.409693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:19.409715 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:19.409818 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:19.410515 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.411916 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.413448 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.414097 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.414224 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:19.416768 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:19.416772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:19.416772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:19.416782 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:19.418307 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:19.418756 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:19.418757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:19.418758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:19.418835 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:19.419083 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:19.419084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:19.419084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:19.419332 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:19.419403 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:19.419669 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 17:05:19.419792 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260808 17:05:19.419891 26969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.409343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:41241 (local address 127.30.194.2:56453)
0808 17:05:19.409436 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:19.409443 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:19.409508 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:19.409859 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:19.409862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:19.410144 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:19.410403 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:19.410410 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.412010 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.412011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:19.413362 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:19.413366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.414208 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.414210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:19.414364 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:19.414948 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:19.414952 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:19.416662 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:19.418385 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:19.418392 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:19.418394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:19.418677 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:19.418899 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:19.418901 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:19.418901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:19.419024 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:19.419395 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:19.419401 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:19.419525 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 17:05:19.419610 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 17:05:19.419782 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":88,"thread_start_us":49,"threads_started":1}
I20260808 17:05:19.419946 26972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.409340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:38673 (local address 127.30.194.3:42443)
0808 17:05:19.409518 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:19.409530 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:19.409591 (+ 61us) server_negotiation.cc:407] Connection header received
0808 17:05:19.410108 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:19.410111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:19.410152 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:19.410252 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:19.410567 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.411917 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.413448 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.414097 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.414223 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:19.417009 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:19.417013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:19.417013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:19.417024 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:19.418754 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:19.419017 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:19.419019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:19.419020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:19.419103 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:19.419272 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:19.419273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:19.419274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:19.419484 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:19.419550 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:19.419714 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 17:05:19.419841 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":64,"threads_started":1}
I20260808 17:05:19.420275 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.409234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:42443 (local address 127.30.194.2:38673)
0808 17:05:19.409277 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:19.409308 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:19.409508 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:19.410286 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:19.410288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:19.410433 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:19.410485 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:19.410487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.412022 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.412023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:19.413362 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:19.413366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.414211 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.414212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:19.414366 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:19.414920 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:19.414931 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:19.416909 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:19.418805 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:19.418805 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:19.418806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:19.418955 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:19.419158 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:19.419158 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:19.419159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:19.419217 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:19.419544 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:19.419545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:19.419590 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 17:05:19.419653 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 17:05:19.419853 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":157}
I20260808 17:05:19.420506 26975 raft_consensus.cc:482] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:05:19.420519 26465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb475cd195946b2a9e95db8ec08aa21" candidate_uuid: "cc9ba4a566624e7c8f32616bf133b301" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" is_pre_election: true
I20260808 17:05:19.420609 26975 raft_consensus.cc:504] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:05:19.420760 26465 raft_consensus.cc:2454] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9ba4a566624e7c8f32616bf133b301 in term 1.
I20260808 17:05:19.420863 26975 leader_election.cc:288] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc9ba4a566624e7c8f32616bf133b301 (127.30.194.2:39323), 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443)
W20260808 17:05:19.421247 26569 leader_election.cc:341] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443): Not found: Tablet not found: ffb475cd195946b2a9e95db8ec08aa21
I20260808 17:05:19.421339 26569 leader_election.cc:302] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb3686e4e6a845fbafa65fdc1b9be694, cc9ba4a566624e7c8f32616bf133b301; no voters: 7766a88f15b3404a911286aca42da04c
I20260808 17:05:19.421507 26966 raft_consensus.cc:2781] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 17:05:19.421568 26966 raft_consensus.cc:482] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:05:19.421595 26966 raft_consensus.cc:3037] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:05:19.422914 26966 raft_consensus.cc:504] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:05:19.423030 26966 leader_election.cc:288] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [CANDIDATE]: Term 2 election: Requested vote from peers 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443), bb3686e4e6a845fbafa65fdc1b9be694 (127.30.194.1:41241)
I20260808 17:05:19.423272 26465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb475cd195946b2a9e95db8ec08aa21" candidate_uuid: "cc9ba4a566624e7c8f32616bf133b301" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bb3686e4e6a845fbafa65fdc1b9be694"
I20260808 17:05:19.423347 26465 raft_consensus.cc:3037] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:05:19.424515 26465 raft_consensus.cc:2454] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9ba4a566624e7c8f32616bf133b301 in term 2.
W20260808 17:05:19.424671 26569 leader_election.cc:341] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443): Not found: Tablet not found: ffb475cd195946b2a9e95db8ec08aa21
I20260808 17:05:19.424734 26569 leader_election.cc:302] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb3686e4e6a845fbafa65fdc1b9be694, cc9ba4a566624e7c8f32616bf133b301; no voters: 7766a88f15b3404a911286aca42da04c
I20260808 17:05:19.424808 26966 raft_consensus.cc:2781] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:05:19.424932 26966 raft_consensus.cc:686] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [term 2 LEADER]: Becoming Leader. State: Replica: cc9ba4a566624e7c8f32616bf133b301, State: Running, Role: LEADER
I20260808 17:05:19.425045 26966 consensus_queue.cc:237] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:05:19.425345 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.425261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:46441)
0808 17:05:19.425297 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:19.425320 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
Aug 08 17:05:19 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.2@KRBTEST.COM
Aug 08 17:05:19 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.3@KRBTEST.COM
I20260808 17:05:19.429241 26976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.420987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:39323 (local address 127.30.194.1:57539)
0808 17:05:19.421192 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:19.421202 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:19.421284 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:19.421494 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:19.421494 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:19.421784 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:19.421842 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:19.421844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.422931 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.422932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:19.423712 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:19.423715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.424297 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.424298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:19.424371 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:19.425173 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:19.425178 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:19.426655 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:19.428187 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:19.428193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:19.428195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:19.428382 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:19.428579 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:19.428580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:19.428580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:19.428655 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:19.428898 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:19.428904 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:19.428951 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 17:05:19.429023 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:05:19.429151 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":4,"thread_start_us":56,"threads_started":1}
I20260808 17:05:19.429239 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.421049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:57539 (local address 127.30.194.2:39323)
0808 17:05:19.421167 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:19.421172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:19.421256 (+ 84us) server_negotiation.cc:407] Connection header received
0808 17:05:19.421356 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:19.421360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:19.421372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:19.421440 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:19.421922 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.422858 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.423791 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.424211 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.424288 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:19.426729 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:19.426736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:19.426736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:19.426743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:19.428115 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:19.428464 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:19.428466 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:19.428467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:19.428524 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:19.428704 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:19.428705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:19.428705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:19.428836 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:19.428904 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:19.429061 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 17:05:19.429151 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":69,"threads_started":1}
I20260808 17:05:19.429337 26972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.421284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:58421 (local address 127.30.194.3:42443)
0808 17:05:19.421316 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:19.421317 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:19.421324 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:05:19.421340 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:19.421341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:19.421349 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:19.421405 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:19.422961 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.423550 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.424379 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.424609 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.424664 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:19.427149 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:19.427150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:19.427150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:19.427158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:19.428357 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:19.428654 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:19.428655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:19.428655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:19.428708 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:19.428882 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:19.428883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:19.428884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:19.428981 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:19.429033 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:19.429177 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 17:05:19.429254 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:05:19.429344 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.421155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:42443 (local address 127.30.194.1:58421)
0808 17:05:19.421237 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:19.421245 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:19.421284 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:19.421408 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:19.421414 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:19.421785 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:19.421842 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:19.421844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.423616 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.423618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:19.424323 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:19.424326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:19.424713 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:19.424714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:19.424773 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:19.425173 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:19.425178 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:19.427067 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:19.428427 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:19.428428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:19.428429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:19.428591 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:19.428770 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:19.428771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:19.428772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:19.428832 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:19.429033 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:19.429035 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:19.429075 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 17:05:19.429155 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 17:05:19.429252 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":4,"thread_start_us":47,"threads_started":1}
I20260808 17:05:19.430013 26678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb475cd195946b2a9e95db8ec08aa21" candidate_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cc9ba4a566624e7c8f32616bf133b301" is_pre_election: true
W20260808 17:05:19.430088 26359 leader_election.cc:341] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443): Not found: Tablet not found: ffb475cd195946b2a9e95db8ec08aa21
I20260808 17:05:19.430337 26359 leader_election.cc:302] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb3686e4e6a845fbafa65fdc1b9be694; no voters: 7766a88f15b3404a911286aca42da04c, cc9ba4a566624e7c8f32616bf133b301
I20260808 17:05:19.430446 26975 raft_consensus.cc:2726] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:05:19.857455 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:19.857329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:34625)
0808 17:05:19.857382 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:19.857417 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 17:05:20.015801 26569 consensus_peers.cc:487] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 -> Peer 7766a88f15b3404a911286aca42da04c (127.30.194.3:42443): Couldn't send request to peer 7766a88f15b3404a911286aca42da04c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffb475cd195946b2a9e95db8ec08aa21. This is attempt 1: this message will repeat every 5th retry.
I20260808 17:05:20.042474 26465 raft_consensus.cc:1261] T ffb475cd195946b2a9e95db8ec08aa21 P bb3686e4e6a845fbafa65fdc1b9be694 [term 2 FOLLOWER]: Refusing update from remote peer cc9ba4a566624e7c8f32616bf133b301: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260808 17:05:20.042814 26982 consensus_queue.cc:1035] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260808 17:05:20.043618 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:20.043504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:42149)
0808 17:05:20.043549 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:20.043584 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 17:05:20.044534 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:20.044350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:43527)
0808 17:05:20.044454 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:20.044491 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":38,"threads_started":1}
I20260808 17:05:20.044649 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:20.044551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:39349)
0808 17:05:20.044584 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:20.044614 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 17:05:20.059782 27000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:20.059618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:37175)
0808 17:05:20.059722 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:20.059748 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
I20260808 17:05:20.562284 27001 ts_tablet_manager.cc:927] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Initiating tablet copy from peer cc9ba4a566624e7c8f32616bf133b301 (127.30.194.2:39323)
I20260808 17:05:20.562513 27001 tablet_copy_client.cc:316] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: tablet copy: Beginning tablet copy session from remote peer at address 127.30.194.2:39323
Aug 08 17:05:20 dist-test-slave-2j7r krb5kdc[24523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.2@KRBTEST.COM
I20260808 17:05:20.569809 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:20.562874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:53121 (local address 127.30.194.2:39323)
0808 17:05:20.562991 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:20.562993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:20.563041 (+ 48us) server_negotiation.cc:407] Connection header received
0808 17:05:20.563115 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:20.563116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:20.563129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:20.563190 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:20.563686 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:20.564358 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:20.565176 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:20.565388 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:20.565444 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:20.567446 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:20.567448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:20.567448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:20.567454 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:20.568662 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:20.569067 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:20.569068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:20.569069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:20.569151 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:20.569365 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:20.569365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:20.569366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:20.569466 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:20.569510 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:20.569670 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 17:05:20.569729 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260808 17:05:20.569832 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:20.562786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:39323 (local address 127.30.194.3:53121)
0808 17:05:20.562957 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:20.562977 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:20.563061 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:20.563200 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:20.563205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:20.563522 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:20.563589 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:20.563591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:20.564440 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:20.564441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:20.565079 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:20.565110 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:20.565449 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:20.565450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:20.565539 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:20.565984 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:20.565988 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:20.567365 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:20.568741 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:20.568751 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:20.568753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:20.569005 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:20.569220 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:20.569221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:20.569221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:20.569294 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:20.569519 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:20.569526 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:20.569576 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 17:05:20.569650 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 17:05:20.569747 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":108,"threads_started":1}
I20260808 17:05:20.570151 26707 tablet_copy_service.cc:140] P cc9ba4a566624e7c8f32616bf133b301: Received BeginTabletCopySession request for tablet ffb475cd195946b2a9e95db8ec08aa21 from peer 7766a88f15b3404a911286aca42da04c ({username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:53121)
I20260808 17:05:20.570220 26707 tablet_copy_service.cc:161] P cc9ba4a566624e7c8f32616bf133b301: Beginning new tablet copy session on tablet ffb475cd195946b2a9e95db8ec08aa21 from peer 7766a88f15b3404a911286aca42da04c at {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:53121: session id = 7766a88f15b3404a911286aca42da04c-ffb475cd195946b2a9e95db8ec08aa21
I20260808 17:05:20.570695 26707 tablet_copy_source_session.cc:205] T ffb475cd195946b2a9e95db8ec08aa21 P cc9ba4a566624e7c8f32616bf133b301: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 17:05:20.571699 27001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb475cd195946b2a9e95db8ec08aa21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:20.574295 27001 tablet_copy_client.cc:791] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: tablet copy: Starting download of 0 data blocks...
I20260808 17:05:20.574458 27001 tablet_copy_client.cc:655] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: tablet copy: Starting download of 1 WAL segments...
I20260808 17:05:20.575338 27001 tablet_copy_client.cc:531] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 17:05:20.576007 27001 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Bootstrap starting.
I20260808 17:05:20.577373 27001 log.cc:824] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:20.579648 27001 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:05:20.579849 27001 tablet_bootstrap.cc:492] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Bootstrap complete.
I20260808 17:05:20.579963 27001 ts_tablet_manager.cc:1397] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 17:05:20.580936 27001 raft_consensus.cc:348] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:05:20.581079 27001 raft_consensus.cc:729] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7766a88f15b3404a911286aca42da04c, State: Initialized, Role: FOLLOWER
I20260808 17:05:20.581259 27001 consensus_queue.cc:260] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9ba4a566624e7c8f32616bf133b301" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39323 } } peers { permanent_uuid: "7766a88f15b3404a911286aca42da04c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 42443 } } peers { permanent_uuid: "bb3686e4e6a845fbafa65fdc1b9be694" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 41241 } }
I20260808 17:05:20.581501 27001 ts_tablet_manager.cc:1428] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20260808 17:05:20.581772 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:20.581662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:49805)
0808 17:05:20.581695 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:20.581727 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 17:05:20.581768 26707 tablet_copy_service.cc:342] P cc9ba4a566624e7c8f32616bf133b301: Request end of tablet copy session 7766a88f15b3404a911286aca42da04c-ffb475cd195946b2a9e95db8ec08aa21 received from {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:53121
I20260808 17:05:20.581811 26707 tablet_copy_service.cc:433] P cc9ba4a566624e7c8f32616bf133b301: ending tablet copy session 7766a88f15b3404a911286aca42da04c-ffb475cd195946b2a9e95db8ec08aa21 on tablet ffb475cd195946b2a9e95db8ec08aa21 with peer 7766a88f15b3404a911286aca42da04c
I20260808 17:05:20.582056 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:20.581971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:35625)
0808 17:05:20.582006 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:20.582031 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 17:05:21.045079 27008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:21.044882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:47927)
0808 17:05:21.045006 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:21.045035 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260808 17:05:21.045190 27010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:21.044979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:57785)
0808 17:05:21.045104 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:21.045139 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260808 17:05:21.064486 27008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:21.064343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:43463)
0808 17:05:21.064406 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:21.064447 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 17:05:21.064522 26889 raft_consensus.cc:1203] T ffb475cd195946b2a9e95db8ec08aa21 P 7766a88f15b3404a911286aca42da04c [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260808 17:05:21.064980 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:21.064882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:51167)
0808 17:05:21.064920 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:21.064947 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 17:05:22.045804 27016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:22.045602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:45401)
0808 17:05:22.045717 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:22.045752 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260808 17:05:22.065140 27018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:22.064934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:51017)
0808 17:05:22.065043 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:22.065079 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260808 17:05:22.065503 27020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:22.065342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:37185)
0808 17:05:22.065437 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:22.065469 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":46,"threads_started":1}
I20260808 17:05:23.046506 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:23.046302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:36713)
0808 17:05:23.046424 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:23.046460 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260808 17:05:23.065804 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:23.065566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:56767)
0808 17:05:23.065702 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:23.065748 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260808 17:05:23.066109 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:23.065911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:43701)
0808 17:05:23.066042 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:23.066074 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260808 17:05:23.066231 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:23.066113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:53855)
0808 17:05:23.066161 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:23.066200 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 17:05:24.047097 27024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:24.046902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:56863)
0808 17:05:24.047017 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:24.047050 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":64,"threads_started":1}
I20260808 17:05:24.066676 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:24.066476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:42851)
0808 17:05:24.066610 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:24.066639 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260808 17:05:24.066811 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:24.066623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:52739)
0808 17:05:24.066725 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:24.066755 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":47,"threads_started":1}
I20260808 17:05:25.047660 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:25.047468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:33041)
0808 17:05:25.047584 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:25.047615 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260808 17:05:25.067283 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:25.067057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:47747)
0808 17:05:25.067201 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:25.067236 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260808 17:05:25.067346 27031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:25.067189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:55075)
0808 17:05:25.067279 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:25.067305 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":41,"threads_started":1}
I20260808 17:05:26.048274 27033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:26.048053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:51063)
0808 17:05:26.048192 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:26.048228 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260808 17:05:26.067890 27038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:26.067679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:56677)
0808 17:05:26.067815 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:26.067847 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":56,"threads_started":1}
I20260808 17:05:26.067898 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:26.067679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:33985)
0808 17:05:26.067818 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:26.067851 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":59,"threads_started":1}
I20260808 17:05:27.048875 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:27.048636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:59685)
0808 17:05:27.048782 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:27.048824 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":59,"threads_started":1}
I20260808 17:05:27.068511 27045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:27.068333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:55853)
0808 17:05:27.068443 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:27.068471 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":46,"threads_started":1}
I20260808 17:05:27.068523 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:27.068308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:60705)
0808 17:05:27.068448 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:27.068485 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260808 17:05:28.049510 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:28.049290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:50561)
0808 17:05:28.049428 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:28.049462 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260808 17:05:28.069128 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:28.068902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:54551)
0808 17:05:28.069032 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:28.069060 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260808 17:05:28.069140 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:28.068932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:37307)
0808 17:05:28.069045 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:28.069074 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260808 17:05:29.050124 27052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:29.049931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:56565)
0808 17:05:29.050043 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:29.050076 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260808 17:05:29.069739 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:29.069508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:50927)
0808 17:05:29.069648 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:29.069691 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260808 17:05:29.069800 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:29.069547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:37489)
0808 17:05:29.069715 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:29.069752 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260808 17:05:30.050704 27055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:30.050484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:40691)
0808 17:05:30.050623 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:30.050652 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260808 17:05:30.070376 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:30.070145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:47053)
0808 17:05:30.070304 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:30.070339 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":64,"threads_started":1}
I20260808 17:05:30.070382 27056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:30.070145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:39183)
0808 17:05:30.070312 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:30.070339 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260808 17:05:31.051344 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:31.051140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:44419)
0808 17:05:31.051266 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:31.051297 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260808 17:05:31.070961 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:31.070746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:60851)
0808 17:05:31.070889 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:31.070921 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260808 17:05:31.070973 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:31.070745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:57895)
0808 17:05:31.070904 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:31.070935 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260808 17:05:32.051921 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:32.051719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:47669)
0808 17:05:32.051832 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:32.051872 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260808 17:05:32.071514 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:32.071320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:49541)
0808 17:05:32.071439 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:32.071475 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260808 17:05:32.071533 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:32.071353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:46225)
0808 17:05:32.071463 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:32.071496 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260808 17:05:33.052625 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:33.052420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.1:52113)
0808 17:05:33.052544 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:33.052579 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260808 17:05:33.063633 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 26276
I20260808 17:05:33.073118 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:33.072047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.2:57275)
0808 17:05:33.073023 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:33.073055 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":961,"thread_start_us":891,"threads_started":1}
I20260808 17:05:33.073138 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:33.072117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:43273 (local address 127.30.194.3:37181)
0808 17:05:33.073015 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:33.073060 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.62:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":832,"threads_started":1}
I20260808 17:05:33.077023 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 26539
I20260808 17:05:33.087646 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 26756
2026-08-08T17:05:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58394 ms)
[----------] 30 tests from SecurityITest (231748 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:33 dist-test-slave-2j7r krb5kdc[27074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:33 dist-test-slave-2j7r krb5kdc[27074](info): set up 2 sockets
Aug 08 17:05:33 dist-test-slave-2j7r krb5kdc[27074](info): commencing operation
krb5kdc: starting...
W20260808 17:05:34.235350 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:34 dist-test-slave-2j7r krb5kdc[27074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:05:34.345820 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:33031
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:33031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:34.425726 27095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:34.425922 27095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:34.425968 27095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:34.427074 27095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:34.427112 27095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:34.427131 27095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:34.427145 27095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:34.427162 27095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:34.428596 27095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:33031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:33031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:34.428829 27095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:34.429019 27095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:34.433662 27103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:34.433687 27095 server_base.cc:1034] running on GCE node
W20260808 17:05:34.433785 27109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:34.433843 27104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:34.434207 27095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:34.434507 27095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:34.435663 27095 hybrid_clock.cc:648] HybridClock initialized: now 1786208734435641 us; error 32 us; skew 500 ppm
Aug 08 17:05:34 dist-test-slave-2j7r krb5kdc[27074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208734, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:34.438040 27095 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:05:34.438469 27095 webserver.cc:460] Webserver started at http://127.30.194.62:35465/ using document root <none> and password file <none>
I20260808 17:05:34.438655 27095 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:34.438701 27095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:34.438829 27095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:34.439779 27095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "c1f7385feee54a6e976a474d59b9ac74"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "66836cf0c4c1a387be83e0fce05b1267"
server_key_iv: "4b1e43b323d9e367142c43b2939b7a23"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.440089 27095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "c1f7385feee54a6e976a474d59b9ac74"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "66836cf0c4c1a387be83e0fce05b1267"
server_key_iv: "4b1e43b323d9e367142c43b2939b7a23"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.441303 27095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:05:34.441958 27117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:34.442133 27095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:34.442229 27095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "c1f7385feee54a6e976a474d59b9ac74"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "66836cf0c4c1a387be83e0fce05b1267"
server_key_iv: "4b1e43b323d9e367142c43b2939b7a23"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.442291 27095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:34.452042 27095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:34.453812 27095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:34.453938 27095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:34.457598 27095 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:33031
I20260808 17:05:34.457628 27213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:33031 every 8 connection(s)
I20260808 17:05:34.457926 27095 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:34.458721 27214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:34.459976 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 27095
I20260808 17:05:34.460075 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:34.460242 31496 external_mini_cluster.cc:1428] Setting key 4ca946daeeeb89ad94a9cad6ca71384d
I20260808 17:05:34.461046 27214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74: Bootstrap starting.
I20260808 17:05:34.461661 27214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:34.461974 27214 log.cc:824] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:34.462664 27214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74: No bootstrap required, opened a new log
I20260808 17:05:34.464381 27214 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f7385feee54a6e976a474d59b9ac74" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33031 } }
I20260808 17:05:34.464509 27214 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:34.464540 27214 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f7385feee54a6e976a474d59b9ac74, State: Initialized, Role: FOLLOWER
I20260808 17:05:34.464661 27214 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f7385feee54a6e976a474d59b9ac74" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33031 } }
I20260808 17:05:34.464738 27214 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:34.464769 27214 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:34.464815 27214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:34.465365 27214 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f7385feee54a6e976a474d59b9ac74" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33031 } }
I20260808 17:05:34.465461 27214 leader_election.cc:302] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1f7385feee54a6e976a474d59b9ac74; no voters:
I20260808 17:05:34.465612 27214 leader_election.cc:288] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:34.465658 27226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:34.465801 27226 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [term 1 LEADER]: Becoming Leader. State: Replica: c1f7385feee54a6e976a474d59b9ac74, State: Running, Role: LEADER
I20260808 17:05:34.465889 27214 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:34.465946 27226 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f7385feee54a6e976a474d59b9ac74" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33031 } }
Aug 08 17:05:34 dist-test-slave-2j7r krb5kdc[27074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:34.466295 27227 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1f7385feee54a6e976a474d59b9ac74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f7385feee54a6e976a474d59b9ac74" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33031 } } }
I20260808 17:05:34.466361 27227 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:34.466537 27228 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1f7385feee54a6e976a474d59b9ac74. Latest consensus state: current_term: 1 leader_uuid: "c1f7385feee54a6e976a474d59b9ac74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f7385feee54a6e976a474d59b9ac74" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33031 } } }
I20260808 17:05:34.466610 27228 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:34.466904 27235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:34.467432 27235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:34.469489 27235 catalog_manager.cc:1347] Generated new cluster ID: bc9cf05c71244964b245e5dc9df33865
I20260808 17:05:34.469534 27235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:34.470003 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.460728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46338 (local address 127.30.194.62:33031)
0808 17:05:34.460882 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:34.460890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:34.460914 (+ 24us) server_negotiation.cc:407] Connection header received
0808 17:05:34.461354 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:34.461361 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:34.461429 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:34.461540 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:34.462302 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.463199 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.463904 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.464282 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.464381 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.466846 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:34.466851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:34.466851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:34.466863 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:34.468559 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:34.468884 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:34.468888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:34.468888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:34.468982 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:34.469172 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:34.469173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:34.469174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:34.469443 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:34.469578 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:34.469680 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 17:05:34.469809 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260808 17:05:34.476774 27235 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:34.477478 27235 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:34.490366 27235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74: Generated new TSK 0
I20260808 17:05:34.490554 27235 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:05:34.515348 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33031
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:34.594455 27260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:34.594635 27260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:34.594656 27260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:34.595697 27260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:34.595738 27260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:34.595762 27260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:05:34.597131 27260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:34.597296 27260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:34.597471 27260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:34.601955 27270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:34.601990 27265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:34.602001 27266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:34.602128 27260 server_base.cc:1034] running on GCE node
I20260808 17:05:34.602380 27260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:34.602593 27260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:34.603727 27260 hybrid_clock.cc:648] HybridClock initialized: now 1786208734603718 us; error 26 us; skew 500 ppm
Aug 08 17:05:34 dist-test-slave-2j7r krb5kdc[27074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208734, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:34.606135 27260 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:05:34.606500 27260 webserver.cc:460] Webserver started at http://127.30.194.1:45383/ using document root <none> and password file <none>
I20260808 17:05:34.606683 27260 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:34.606725 27260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:34.606813 27260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:34.607731 27260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "6b9196eebf56402fbd5a1ae0af6eaf78"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "191b16939173da392840fed8d64f043e"
server_key_iv: "af07d0bcfac9c6889dd5dc56a415dc70"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.608009 27260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "6b9196eebf56402fbd5a1ae0af6eaf78"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "191b16939173da392840fed8d64f043e"
server_key_iv: "af07d0bcfac9c6889dd5dc56a415dc70"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.609184 27260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:05:34.609938 27278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:34.610111 27260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:05:34.610195 27260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "6b9196eebf56402fbd5a1ae0af6eaf78"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "191b16939173da392840fed8d64f043e"
server_key_iv: "af07d0bcfac9c6889dd5dc56a415dc70"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.610267 27260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:34.630231 27260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:34.631845 27260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:34.631963 27260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:34.632176 27260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:34.632462 27260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:34.632493 27260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:34.632513 27260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:34.632536 27260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:34.637944 27260 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:35607
I20260808 17:05:34.638087 27478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:35607 every 8 connection(s)
I20260808 17:05:34.638325 27260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:05:34.639657 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 27260
I20260808 17:05:34.639747 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:05:34.639923 31496 external_mini_cluster.cc:1428] Setting key 33313cb9bb59f013026ad4f2fc652e14
Aug 08 17:05:34 dist-test-slave-2j7r krb5kdc[27074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208734, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:34.647300 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.638968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:39341 (local address 127.30.194.62:33031)
0808 17:05:34.639016 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:34.639019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:34.639214 (+ 195us) server_negotiation.cc:407] Connection header received
0808 17:05:34.639675 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:34.639677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:34.639690 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:34.639740 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:34.640787 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.641315 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.642254 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.642460 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.642515 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.644650 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:34.644651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:34.644651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:34.644658 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:34.645983 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:34.646435 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:34.646436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:34.646437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:34.646488 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:34.646736 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:34.646737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:34.646737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:34.646827 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:34.646871 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:34.647168 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 17:05:34.647237 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:05:34.647536 27485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.638936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:33031 (local address 127.30.194.1:39341)
0808 17:05:34.639100 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:34.639133 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:34.639576 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:34.639867 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:34.639871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:34.640473 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:34.640695 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.640700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.641400 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.641401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.642173 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.642176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.642527 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.642528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.642646 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.643044 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:34.643053 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:34.644567 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:34.646053 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:34.646061 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:34.646063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:34.646364 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:34.646544 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:34.646545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:34.646545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:34.646678 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:34.646878 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:34.646883 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:34.647037 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 17:05:34.647146 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 17:05:34.647304 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260808 17:05:34.648057 27481 heartbeater.cc:347] Connected to a master server at 127.30.194.62:33031
I20260808 17:05:34.648154 27481 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:34.648344 27481 heartbeater.cc:511] Master 127.30.194.62:33031 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
I20260808 17:05:34.648917 27149 ts_manager.cc:194] Registered new tserver with Master: 6b9196eebf56402fbd5a1ae0af6eaf78 (127.30.194.1:35607)
I20260808 17:05:34.649555 27149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:39341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:05:34.663727 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33031
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:34.742383 27489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:34.742568 27489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:34.742597 27489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:34.743677 27489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:34.743716 27489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:34.743749 27489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:05:34.745167 27489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:34.745378 27489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:34.745577 27489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:34.750233 27503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:34.750219 27505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:34.750211 27501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:34.750415 27489 server_base.cc:1034] running on GCE node
I20260808 17:05:34.750638 27489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:34.750869 27489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:34.752009 27489 hybrid_clock.cc:648] HybridClock initialized: now 1786208734751994 us; error 27 us; skew 500 ppm
Aug 08 17:05:34 dist-test-slave-2j7r krb5kdc[27074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208734, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:34.754311 27489 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:05:34.754691 27489 webserver.cc:460] Webserver started at http://127.30.194.2:33009/ using document root <none> and password file <none>
I20260808 17:05:34.754881 27489 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:34.754928 27489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:34.755052 27489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:34.755995 27489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "ebd7767865054e52ad7311eb98e7b081"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "fcf8218642a433b0cb5e87a8166cd890"
server_key_iv: "4e08d79285d0808a58255b54a4605977"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.756316 27489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "ebd7767865054e52ad7311eb98e7b081"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "fcf8218642a433b0cb5e87a8166cd890"
server_key_iv: "4e08d79285d0808a58255b54a4605977"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.757573 27489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:05:34.758229 27515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:34.758378 27489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:34.758460 27489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "ebd7767865054e52ad7311eb98e7b081"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "fcf8218642a433b0cb5e87a8166cd890"
server_key_iv: "4e08d79285d0808a58255b54a4605977"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.758513 27489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:34.783754 27489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:34.785701 27489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:34.785804 27489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:34.785995 27489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:34.786226 27489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:34.786254 27489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:34.786273 27489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:34.786284 27489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:34.791440 27489 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:37489
I20260808 17:05:34.791504 27701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:37489 every 8 connection(s)
I20260808 17:05:34.791752 27489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:05:34 dist-test-slave-2j7r krb5kdc[27074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208734, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:34.798225 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 27489
I20260808 17:05:34.798338 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:05:34.798532 31496 external_mini_cluster.cc:1428] Setting key d6d20bac688e199ae174ad823c46f2ba
I20260808 17:05:34.800387 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.792399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:48833 (local address 127.30.194.62:33031)
0808 17:05:34.792455 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:34.792457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:34.792642 (+ 185us) server_negotiation.cc:407] Connection header received
0808 17:05:34.793130 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:34.793131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:34.793144 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:34.793196 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:34.794068 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.794550 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.795477 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.795668 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.795731 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.797593 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:34.797595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:34.797595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:34.797601 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:34.799061 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:34.799515 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:34.799516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:34.799516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:34.799576 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:34.799809 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:34.799809 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:34.799809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:34.799911 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:34.799962 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:34.800242 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 17:05:34.800308 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:05:34.800560 27704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.792387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:33031 (local address 127.30.194.2:48833)
0808 17:05:34.792523 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:34.792560 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:34.793012 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:34.793269 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:34.793273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:34.793804 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:34.793971 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.793975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.794630 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.794631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.795392 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.795394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.795734 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.795735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.795852 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.796207 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:34.796215 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:34.797512 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:34.799137 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:34.799143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:34.799145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:34.799436 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:34.799632 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:34.799633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:34.799633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:34.799754 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:34.799962 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:34.799967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:34.800111 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 17:05:34.800212 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 17:05:34.800364 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
I20260808 17:05:34.800968 27702 heartbeater.cc:347] Connected to a master server at 127.30.194.62:33031
I20260808 17:05:34.801080 27702 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:34.801313 27702 heartbeater.cc:511] Master 127.30.194.62:33031 requested a full tablet report, sending...
I20260808 17:05:34.801680 27149 ts_manager.cc:194] Registered new tserver with Master: ebd7767865054e52ad7311eb98e7b081 (127.30.194.2:37489)
I20260808 17:05:34.802030 27149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:48833
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:05:34.821969 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33031
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:34.903568 27709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:34.903746 27709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:34.903767 27709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:34.904908 27709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:34.904947 27709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:34.904973 27709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:05:34.906438 27709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:34.906643 27709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:34.906876 27709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:34.911629 27723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:34.911708 27719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:34.911695 27709 server_base.cc:1034] running on GCE node
W20260808 17:05:34.911635 27721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:34.912003 27709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:34.912236 27709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:34.913370 27709 hybrid_clock.cc:648] HybridClock initialized: now 1786208734913351 us; error 28 us; skew 500 ppm
Aug 08 17:05:34 dist-test-slave-2j7r krb5kdc[27074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208734, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:34.915660 27709 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:05:34.916067 27709 webserver.cc:460] Webserver started at http://127.30.194.3:37049/ using document root <none> and password file <none>
I20260808 17:05:34.916285 27709 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:34.916332 27709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:34.916450 27709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:34.917448 27709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "47b3fda2390d4d33b9b20f4a1213347c"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "55fcff835f0c4a7db9f4ec495acab6f7"
server_key_iv: "b23c05bdf1d15c6222b72105b278df46"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.917748 27709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "47b3fda2390d4d33b9b20f4a1213347c"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "55fcff835f0c4a7db9f4ec495acab6f7"
server_key_iv: "b23c05bdf1d15c6222b72105b278df46"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.918871 27709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:05:34.919466 27738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:34.919601 27709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:34.919682 27709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "47b3fda2390d4d33b9b20f4a1213347c"
format_stamp: "Formatted at 2026-08-08 17:05:34 on dist-test-slave-2j7r"
server_key: "55fcff835f0c4a7db9f4ec495acab6f7"
server_key_iv: "b23c05bdf1d15c6222b72105b278df46"
server_key_version: "encryptionkey@0"
I20260808 17:05:34.919742 27709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:34.946367 27709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:34.948413 27709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:34.948544 27709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:34.948769 27709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:34.949059 27709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:34.949108 27709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:34.949138 27709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:34.949159 27709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:34.954588 27709 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:45165
I20260808 17:05:34.954627 27909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:45165 every 8 connection(s)
I20260808 17:05:34.954906 27709 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:05:34.956173 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 27709
I20260808 17:05:34.956293 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:05:34.956514 31496 external_mini_cluster.cc:1428] Setting key 7fd6d5a97526605793dec66370e09cdd
Aug 08 17:05:34 dist-test-slave-2j7r krb5kdc[27074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208734, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:34.963027 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.955528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:50865 (local address 127.30.194.62:33031)
0808 17:05:34.955597 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:34.955599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:34.955749 (+ 150us) server_negotiation.cc:407] Connection header received
0808 17:05:34.956299 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:34.956300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:34.956312 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:34.956376 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:34.957482 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.957958 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.958831 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.959004 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.959048 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.960868 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:34.960870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:34.960870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:34.960877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:34.962003 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:34.962328 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:34.962328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:34.962329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:34.962382 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:34.962562 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:34.962562 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:34.962563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:34.962663 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:34.962713 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:34.962908 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 17:05:34.962965 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:05:34.963171 27915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.955504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:33031 (local address 127.30.194.3:50865)
0808 17:05:34.955647 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:34.955679 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:34.956198 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:34.956447 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:34.956451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:34.957162 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:34.957405 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.957411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.958019 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.958020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.958774 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.958776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.959058 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.959059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.959147 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.959488 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:34.959495 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:34.960780 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:34.962066 (+ 1286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:34.962072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:34.962073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:34.962273 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:34.962428 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:34.962428 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:34.962429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:34.962514 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:34.962708 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:34.962710 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:34.962810 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 17:05:34.962889 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 17:05:34.962998 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260808 17:05:34.963567 27912 heartbeater.cc:347] Connected to a master server at 127.30.194.62:33031
I20260808 17:05:34.963642 27912 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:34.963802 27912 heartbeater.cc:511] Master 127.30.194.62:33031 requested a full tablet report, sending...
I20260808 17:05:34.964144 27149 ts_manager.cc:194] Registered new tserver with Master: 47b3fda2390d4d33b9b20f4a1213347c (127.30.194.3:45165)
I20260808 17:05:34.964474 27149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:50865
I20260808 17:05:34.968057 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:05:34.974159 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.969291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46340 (local address 127.30.194.62:33031)
0808 17:05:34.969328 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:34.969331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:34.969411 (+ 80us) server_negotiation.cc:407] Connection header received
0808 17:05:34.969458 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:34.969459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:34.969466 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:34.969506 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:34.970039 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.970470 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.971113 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.971274 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.971311 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.972215 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:34.972216 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:34.972217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:34.972222 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:34.973374 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:34.973641 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:34.973642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:34.973642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:34.973682 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:34.973801 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:34.973801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:34.973801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:34.973920 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:34.973984 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:34.974049 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 17:05:34.974100 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:05:34.975502 27149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:05:34.976966 27149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:05:34.982693 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.978748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:37489 (local address 127.0.0.1:55166)
0808 17:05:34.979023 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:34.979066 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:34.979132 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:34.979386 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:34.979390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:34.979392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:34.979453 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.979456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.980704 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.980705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.981793 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.981795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.982421 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.982422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.982499 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.982511 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 17:05:34.982574 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 17:05:34.982624 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":33,"thread_start_us":51,"threads_started":1}
I20260808 17:05:34.982710 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.978972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.30.194.2:37489)
0808 17:05:34.979159 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:34.979164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:34.979173 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:05:34.979210 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:34.979212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:34.979226 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:34.979297 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:34.979534 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.980605 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.981861 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.982346 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.982407 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.982464 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:34.982595 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 17:05:34.982639 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260808 17:05:34.982821 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.978930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:35607 (local address 127.0.0.1:44944)
0808 17:05:34.979260 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:34.979268 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:34.979309 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:34.979458 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:34.979459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:34.979461 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:34.979563 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.979565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.980431 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.980432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.981623 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.981626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.982568 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.982569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.982636 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.982640 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:05:34.982686 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 17:05:34.982729 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":3,"thread_start_us":47,"threads_started":1}
I20260808 17:05:34.983036 27929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.978805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40728 (local address 127.30.194.3:45165)
0808 17:05:34.978940 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:34.978944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:34.979130 (+ 186us) server_negotiation.cc:407] Connection header received
0808 17:05:34.979283 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:34.979286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:34.979301 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:34.979367 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:34.979532 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.980620 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.981700 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.982409 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.982477 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.982529 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:34.982933 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 17:05:34.982974 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260808 17:05:34.983073 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.979071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44944 (local address 127.30.194.1:35607)
0808 17:05:34.979269 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:34.979273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:34.979302 (+ 29us) server_negotiation.cc:407] Connection header received
0808 17:05:34.979379 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:34.979381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:34.979395 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:34.979453 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:34.979641 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.980337 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.981691 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.982409 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.982477 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.982529 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:34.982965 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 17:05:34.983007 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":94,"threads_started":1}
I20260808 17:05:34.983201 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:34.978748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:45165 (local address 127.0.0.1:40728)
0808 17:05:34.979024 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:34.979066 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:34.979162 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:34.979428 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:34.979429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:34.979430 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:34.979467 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.979468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.980722 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.980724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.981618 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:34.981621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:34.982566 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:34.982567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:34.982636 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:34.982641 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:05:34.982686 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:05:34.982729 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":26,"spinlock_wait_cycles":384,"thread_start_us":65,"threads_started":1}
I20260808 17:05:34.983383 27599 tablet_service.cc:1467] Processing CreateTablet for tablet 5bfe4243158a40608dfd78d07d07bfde (DEFAULT_TABLE table=test-table [id=97b391b1f5254e899a54384f75ee8408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:34.983739 27599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bfe4243158a40608dfd78d07d07bfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:34.983817 27810 tablet_service.cc:1467] Processing CreateTablet for tablet 5bfe4243158a40608dfd78d07d07bfde (DEFAULT_TABLE table=test-table [id=97b391b1f5254e899a54384f75ee8408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:34.984126 27810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bfe4243158a40608dfd78d07d07bfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:34.987047 27941 tablet_bootstrap.cc:492] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: Bootstrap starting.
I20260808 17:05:34.987079 27942 tablet_bootstrap.cc:492] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: Bootstrap starting.
I20260808 17:05:34.987537 27368 tablet_service.cc:1467] Processing CreateTablet for tablet 5bfe4243158a40608dfd78d07d07bfde (DEFAULT_TABLE table=test-table [id=97b391b1f5254e899a54384f75ee8408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:34.987737 27941 tablet_bootstrap.cc:654] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:34.987761 27942 tablet_bootstrap.cc:654] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:34.987802 27368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bfe4243158a40608dfd78d07d07bfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:34.988274 27942 log.cc:824] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:34.988274 27941 log.cc:824] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:34.988974 27942 tablet_bootstrap.cc:492] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: No bootstrap required, opened a new log
I20260808 17:05:34.989044 27942 ts_tablet_manager.cc:1397] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:05:34.989274 27941 tablet_bootstrap.cc:492] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: No bootstrap required, opened a new log
I20260808 17:05:34.989341 27941 ts_tablet_manager.cc:1397] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:05:34.990504 27949 tablet_bootstrap.cc:492] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78: Bootstrap starting.
I20260808 17:05:34.991115 27949 tablet_bootstrap.cc:654] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:34.991055 27942 raft_consensus.cc:348] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b3fda2390d4d33b9b20f4a1213347c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45165 } } peers { permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 } } peers { permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 } }
I20260808 17:05:34.991238 27942 raft_consensus.cc:374] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:34.991173 27941 raft_consensus.cc:348] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b3fda2390d4d33b9b20f4a1213347c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45165 } } peers { permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 } } peers { permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 } }
I20260808 17:05:34.991283 27942 raft_consensus.cc:729] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b3fda2390d4d33b9b20f4a1213347c, State: Initialized, Role: FOLLOWER
I20260808 17:05:34.991318 27941 raft_consensus.cc:374] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:34.991353 27941 raft_consensus.cc:729] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebd7767865054e52ad7311eb98e7b081, State: Initialized, Role: FOLLOWER
I20260808 17:05:34.991433 27942 consensus_queue.cc:260] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b3fda2390d4d33b9b20f4a1213347c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45165 } } peers { permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 } } peers { permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 } }
I20260808 17:05:34.991447 27941 consensus_queue.cc:260] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b3fda2390d4d33b9b20f4a1213347c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45165 } } peers { permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 } } peers { permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 } }
I20260808 17:05:34.991611 27949 log.cc:824] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:34.991701 27942 ts_tablet_manager.cc:1428] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 17:05:34.991701 27941 ts_tablet_manager.cc:1428] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:05:34.991815 27702 heartbeater.cc:503] Master 127.30.194.62:33031 was elected leader, sending a full tablet report...
I20260808 17:05:34.991863 27912 heartbeater.cc:503] Master 127.30.194.62:33031 was elected leader, sending a full tablet report...
I20260808 17:05:34.992317 27949 tablet_bootstrap.cc:492] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78: No bootstrap required, opened a new log
I20260808 17:05:34.992383 27949 ts_tablet_manager.cc:1397] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:05:34.993852 27949 raft_consensus.cc:348] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b3fda2390d4d33b9b20f4a1213347c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45165 } } peers { permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 } } peers { permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 } }
I20260808 17:05:34.993976 27949 raft_consensus.cc:374] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:34.994007 27949 raft_consensus.cc:729] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9196eebf56402fbd5a1ae0af6eaf78, State: Initialized, Role: FOLLOWER
I20260808 17:05:34.994110 27949 consensus_queue.cc:260] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b3fda2390d4d33b9b20f4a1213347c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45165 } } peers { permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 } } peers { permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 } }
I20260808 17:05:34.994357 27949 ts_tablet_manager.cc:1428] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:05:34.994366 27481 heartbeater.cc:503] Master 127.30.194.62:33031 was elected leader, sending a full tablet report...
W20260808 17:05:35.042364 27703 tablet.cc:2369] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:05:35.139022 27482 tablet.cc:2369] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:05:35.205472 27913 tablet.cc:2369] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:05:35.374482 27950 raft_consensus.cc:482] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:05:35.374609 27950 raft_consensus.cc:504] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b3fda2390d4d33b9b20f4a1213347c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45165 } } peers { permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 } } peers { permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 } }
I20260808 17:05:35.374976 27950 leader_election.cc:288] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b9196eebf56402fbd5a1ae0af6eaf78 (127.30.194.1:35607), ebd7767865054e52ad7311eb98e7b081 (127.30.194.2:37489)
I20260808 17:05:35.377672 27915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.375197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:35607 (local address 127.30.194.3:33365)
0808 17:05:35.375256 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:35.375269 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:35.375354 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:35.375485 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:35.375486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:35.375487 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:35.375555 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:35.375557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.376163 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.376164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:35.376884 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:35.376886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.377445 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.377446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:35.377517 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:35.377520 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:35.377565 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:05:35.377600 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 17:05:35.377672 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.375293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:33365 (local address 127.30.194.1:35607)
0808 17:05:35.375344 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:35.375346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:35.375356 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:05:35.375412 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:35.375413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:35.375423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:35.375475 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:35.375611 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.376088 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.376966 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.377375 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.377439 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:35.377448 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:35.377576 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 17:05:35.377608 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:05:35.377875 27955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.375218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:37489 (local address 127.30.194.3:52827)
0808 17:05:35.375416 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:35.375425 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:35.375493 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:35.375636 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:35.375636 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:35.375638 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:35.375724 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:35.375726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.376327 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.376328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:35.377066 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:35.377068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.377611 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.377612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:35.377677 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:35.377688 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 17:05:35.377741 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 17:05:35.377781 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":125,"threads_started":1}
I20260808 17:05:35.377995 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.375250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:52827 (local address 127.30.194.2:37489)
0808 17:05:35.375291 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:35.375294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:35.375498 (+ 204us) server_negotiation.cc:407] Connection header received
0808 17:05:35.375559 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:35.375560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:35.375575 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:35.375628 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:35.375789 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.376247 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.377176 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.377530 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.377600 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:35.377608 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:35.377885 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 17:05:35.377926 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:05:35.378007 27398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfe4243158a40608dfd78d07d07bfde" candidate_uuid: "47b3fda2390d4d33b9b20f4a1213347c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" is_pre_election: true
I20260808 17:05:35.378154 27398 raft_consensus.cc:2454] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47b3fda2390d4d33b9b20f4a1213347c in term 0.
I20260808 17:05:35.378294 27632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfe4243158a40608dfd78d07d07bfde" candidate_uuid: "47b3fda2390d4d33b9b20f4a1213347c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebd7767865054e52ad7311eb98e7b081" is_pre_election: true
I20260808 17:05:35.378383 27744 leader_election.cc:302] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b3fda2390d4d33b9b20f4a1213347c, 6b9196eebf56402fbd5a1ae0af6eaf78; no voters:
I20260808 17:05:35.378458 27632 raft_consensus.cc:2454] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47b3fda2390d4d33b9b20f4a1213347c in term 0.
I20260808 17:05:35.378509 27950 raft_consensus.cc:2781] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:05:35.378582 27950 raft_consensus.cc:482] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:05:35.378609 27950 raft_consensus.cc:3037] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:35.379276 27950 raft_consensus.cc:504] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b3fda2390d4d33b9b20f4a1213347c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45165 } } peers { permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 } } peers { permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 } }
I20260808 17:05:35.379405 27950 leader_election.cc:288] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [CANDIDATE]: Term 1 election: Requested vote from peers 6b9196eebf56402fbd5a1ae0af6eaf78 (127.30.194.1:35607), ebd7767865054e52ad7311eb98e7b081 (127.30.194.2:37489)
I20260808 17:05:35.379596 27398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfe4243158a40608dfd78d07d07bfde" candidate_uuid: "47b3fda2390d4d33b9b20f4a1213347c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78"
I20260808 17:05:35.379594 27632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfe4243158a40608dfd78d07d07bfde" candidate_uuid: "47b3fda2390d4d33b9b20f4a1213347c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebd7767865054e52ad7311eb98e7b081"
I20260808 17:05:35.379669 27398 raft_consensus.cc:3037] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:35.379680 27632 raft_consensus.cc:3037] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:35.380476 27398 raft_consensus.cc:2454] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47b3fda2390d4d33b9b20f4a1213347c in term 1.
I20260808 17:05:35.380477 27632 raft_consensus.cc:2454] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47b3fda2390d4d33b9b20f4a1213347c in term 1.
I20260808 17:05:35.380656 27744 leader_election.cc:302] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b3fda2390d4d33b9b20f4a1213347c, 6b9196eebf56402fbd5a1ae0af6eaf78; no voters:
I20260808 17:05:35.380759 27950 raft_consensus.cc:2781] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:35.380899 27950 raft_consensus.cc:686] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 1 LEADER]: Becoming Leader. State: Replica: 47b3fda2390d4d33b9b20f4a1213347c, State: Running, Role: LEADER
I20260808 17:05:35.380987 27950 consensus_queue.cc:237] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b3fda2390d4d33b9b20f4a1213347c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45165 } } peers { permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 } } peers { permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 } }
I20260808 17:05:35.381815 27149 catalog_manager.cc:5495] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c reported cstate change: term changed from 0 to 1, leader changed from <none> to 47b3fda2390d4d33b9b20f4a1213347c (127.30.194.3). New cstate: current_term: 1 leader_uuid: "47b3fda2390d4d33b9b20f4a1213347c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b3fda2390d4d33b9b20f4a1213347c" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 } health_report { overall_health: UNKNOWN } } }
I20260808 17:05:35.419276 27929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.417109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40736 (local address 127.30.194.3:45165)
0808 17:05:35.417154 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:35.417156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:35.417164 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:35.417234 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:35.417235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:35.417253 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:35.417309 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:05:35.417440 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.417886 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.418590 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.418774 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.418826 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:35.418973 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:05:35.419068 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:05:35.419135 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:35.419171 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 17:05:35.419191 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:05:35.420775 27632 raft_consensus.cc:1261] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081 [term 1 FOLLOWER]: Refusing update from remote peer 47b3fda2390d4d33b9b20f4a1213347c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:35.420776 27398 raft_consensus.cc:1261] T 5bfe4243158a40608dfd78d07d07bfde P 6b9196eebf56402fbd5a1ae0af6eaf78 [term 1 FOLLOWER]: Refusing update from remote peer 47b3fda2390d4d33b9b20f4a1213347c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:35.421125 27956 consensus_queue.cc:1035] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebd7767865054e52ad7311eb98e7b081" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 37489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:35.421241 27950 consensus_queue.cc:1035] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b9196eebf56402fbd5a1ae0af6eaf78" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:35.424302 27965 mvcc.cc:205] Tried to move back new op lower bound from 7316310980281339904 to 7316310980121026560. Current Snapshot: MvccSnapshot[applied={T|T < 7316310980281339904 or (T in {7316310980281339904})}]
I20260808 17:05:35.425560 27143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:35.425638 27143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:35.426316 27143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1f7385feee54a6e976a474d59b9ac74: Sending DeleteTablet for 3 replicas of tablet 5bfe4243158a40608dfd78d07d07bfde
I20260808 17:05:35.426614 27810 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bfe4243158a40608dfd78d07d07bfde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:35 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:40728
I20260808 17:05:35.426683 27368 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bfe4243158a40608dfd78d07d07bfde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:35 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:44944
I20260808 17:05:35.426832 27978 tablet_replica.cc:330] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: stopping tablet replica
I20260808 17:05:35.426815 27599 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bfe4243158a40608dfd78d07d07bfde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:35 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:55166
I20260808 17:05:35.426915 27978 raft_consensus.cc:2229] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:05:35.426997 27978 raft_consensus.cc:2258] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:35.427222 27980 tablet_replica.cc:330] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: stopping tablet replica
I20260808 17:05:35.427304 27980 raft_consensus.cc:2229] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:35.427387 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 27260
I20260808 17:05:35.427412 27980 raft_consensus.cc:2258] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:35.427407 27978 ts_tablet_manager.cc:1905] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:35.433030 27980 ts_tablet_manager.cc:1905] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:35.435838 27978 ts_tablet_manager.cc:1918] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:35.435904 27978 log.cc:1196] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/5bfe4243158a40608dfd78d07d07bfde
I20260808 17:05:35.436102 27978 ts_tablet_manager.cc:1939] T 5bfe4243158a40608dfd78d07d07bfde P 47b3fda2390d4d33b9b20f4a1213347c: Deleting consensus metadata
I20260808 17:05:35.436442 27120 catalog_manager.cc:4841] TS 47b3fda2390d4d33b9b20f4a1213347c (127.30.194.3:45165): tablet 5bfe4243158a40608dfd78d07d07bfde (table test-table [id=97b391b1f5254e899a54384f75ee8408]) successfully deleted
I20260808 17:05:35.437193 27980 ts_tablet_manager.cc:1918] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:35.437254 27980 log.cc:1196] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/5bfe4243158a40608dfd78d07d07bfde
I20260808 17:05:35.437462 27980 ts_tablet_manager.cc:1939] T 5bfe4243158a40608dfd78d07d07bfde P ebd7767865054e52ad7311eb98e7b081: Deleting consensus metadata
I20260808 17:05:35.437793 27119 catalog_manager.cc:4841] TS ebd7767865054e52ad7311eb98e7b081 (127.30.194.2:37489): tablet 5bfe4243158a40608dfd78d07d07bfde (table test-table [id=97b391b1f5254e899a54384f75ee8408]) successfully deleted
W20260808 17:05:35.441546 27120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:35607 (error 108)
I20260808 17:05:35.441927 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 27489
I20260808 17:05:35.441967 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.441847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:35607 (local address 127.0.0.1:44946)
0808 17:05:35.441901 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:35.441936 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:35607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 17:05:35.444747 27120 catalog_manager.cc:4584] TS 6b9196eebf56402fbd5a1ae0af6eaf78 (127.30.194.1:35607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5bfe4243158a40608dfd78d07d07bfde: Network error: Client connection negotiation failed: client connection to 127.30.194.1:35607: connect: Connection refused (error 111)
I20260808 17:05:35.454463 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 27709
I20260808 17:05:35.463327 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 27095
2026-08-08T17:05:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2376 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T17:05:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:35Z Disabled control of system clock
I20260808 17:05:35.495332 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40247
--webserver_interface=127.30.194.62
--webserver_port=0
--builtin_ntp_servers=127.30.194.20:34557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:40247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:05:35.577154 27989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:35.577369 27989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:35.577390 27989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:35.578522 27989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:35.578567 27989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:35.578581 27989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:35.578596 27989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:35.578624 27989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:35.579980 27989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:40247
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:35.580230 27989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:35.580513 27989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:35.584897 28000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:35.584997 27997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:35.584880 28002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:35.585125 27989 server_base.cc:1034] running on GCE node
I20260808 17:05:35.585346 27989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:35.585628 27989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:35.586776 27989 hybrid_clock.cc:648] HybridClock initialized: now 1786208735586761 us; error 26 us; skew 500 ppm
I20260808 17:05:35.587859 27989 webserver.cc:460] Webserver started at http://127.30.194.62:38805/ using document root <none> and password file <none>
I20260808 17:05:35.588066 27989 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:35.588112 27989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:35.588238 27989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:35.589277 27989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "fa828a980aa847379166c5e554253ab9"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "04314308c7c945701f8095fcf6c9d341"
server_key_iv: "f94be2d8a371cdc9f09d17d7261013b8"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.589586 27989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "fa828a980aa847379166c5e554253ab9"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "04314308c7c945701f8095fcf6c9d341"
server_key_iv: "f94be2d8a371cdc9f09d17d7261013b8"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.590700 27989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:05:35.591310 28010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:35.591466 27989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:35.591545 27989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "fa828a980aa847379166c5e554253ab9"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "04314308c7c945701f8095fcf6c9d341"
server_key_iv: "f94be2d8a371cdc9f09d17d7261013b8"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.591602 27989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:35.611951 27989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:35.612206 27989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:35.612339 27989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:35.616057 27989 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:40247
I20260808 17:05:35.616091 28093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:40247 every 8 connection(s)
I20260808 17:05:35.616381 27989 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:35.617334 28094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:35.619400 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 27989
I20260808 17:05:35.619503 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:35.619557 28094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9: Bootstrap starting.
I20260808 17:05:35.619675 31496 external_mini_cluster.cc:1428] Setting key 2e1b6922ede36f5a35aabfd6dce3f96b
I20260808 17:05:35.620239 28094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:35.620550 28094 log.cc:824] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:35.621307 28094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9: No bootstrap required, opened a new log
I20260808 17:05:35.623404 28094 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa828a980aa847379166c5e554253ab9" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40247 } }
I20260808 17:05:35.623548 28094 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:35.623589 28094 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa828a980aa847379166c5e554253ab9, State: Initialized, Role: FOLLOWER
I20260808 17:05:35.623703 28094 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa828a980aa847379166c5e554253ab9" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40247 } }
I20260808 17:05:35.623781 28094 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:35.623816 28094 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:35.623857 28094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:35.624475 28094 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa828a980aa847379166c5e554253ab9" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40247 } }
I20260808 17:05:35.624578 28094 leader_election.cc:302] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa828a980aa847379166c5e554253ab9; no voters:
I20260808 17:05:35.624748 28094 leader_election.cc:288] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:35.624786 28101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:35.624941 28101 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [term 1 LEADER]: Becoming Leader. State: Replica: fa828a980aa847379166c5e554253ab9, State: Running, Role: LEADER
I20260808 17:05:35.625005 28094 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:35.625056 28101 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa828a980aa847379166c5e554253ab9" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40247 } }
I20260808 17:05:35.625381 28101 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa828a980aa847379166c5e554253ab9. Latest consensus state: current_term: 1 leader_uuid: "fa828a980aa847379166c5e554253ab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa828a980aa847379166c5e554253ab9" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40247 } } }
I20260808 17:05:35.625465 28101 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:35.625396 28104 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa828a980aa847379166c5e554253ab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa828a980aa847379166c5e554253ab9" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40247 } } }
I20260808 17:05:35.625607 28104 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:35.625636 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.620134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42642 (local address 127.30.194.62:40247)
0808 17:05:35.620332 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:35.620342 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:35.620377 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:05:35.620869 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:35.620876 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:35.620934 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:35.621075 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:35.621273 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.622620 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.623510 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.624084 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.624202 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:35.624767 (+ 565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:35.624771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:35.624771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:35.624964 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:35.625001 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:35.625015 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:35.625292 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:35.625324 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 17:05:35.625398 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260808 17:05:35.625675 28117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:35.626148 28117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:35.627627 28117 catalog_manager.cc:1347] Generated new cluster ID: 70ae346e5a05406b89581213f70573e7
I20260808 17:05:35.627681 28117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:35.651647 28117 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:35.652308 28117 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:35.661728 28117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9: Generated new TSK 0
I20260808 17:05:35.661892 28117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:35.671665 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40247
--builtin_ntp_servers=127.30.194.20:34557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:05:35.751379 28138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:35.751574 28138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:35.751595 28138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:35.752722 28138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:35.752759 28138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:35.752784 28138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:05:35.754109 28138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:35.754328 28138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:35.754588 28138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:35.759436 28146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:35.759469 28148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:35.759498 28138 server_base.cc:1034] running on GCE node
W20260808 17:05:35.759461 28150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:35.759809 28138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:35.760041 28138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:35.761132 28138 hybrid_clock.cc:648] HybridClock initialized: now 1786208735761118 us; error 32 us; skew 500 ppm
I20260808 17:05:35.762338 28138 webserver.cc:460] Webserver started at http://127.30.194.1:40193/ using document root <none> and password file <none>
I20260808 17:05:35.762560 28138 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:35.762609 28138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:35.762737 28138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:35.763715 28138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "9604baf931b242e58346c626009357ab"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "b8fe13c69eeed4b526e15d92ee083150"
server_key_iv: "c8241adc6878eaf0cea7663e5b3b4874"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.764039 28138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "9604baf931b242e58346c626009357ab"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "b8fe13c69eeed4b526e15d92ee083150"
server_key_iv: "c8241adc6878eaf0cea7663e5b3b4874"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.765260 28138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:05:35.765887 28158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:35.766036 28138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:35.766124 28138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "9604baf931b242e58346c626009357ab"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "b8fe13c69eeed4b526e15d92ee083150"
server_key_iv: "c8241adc6878eaf0cea7663e5b3b4874"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.766181 28138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:35.776520 28138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:35.776789 28138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:35.776923 28138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:35.777169 28138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:35.777494 28138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:35.777530 28138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:35.777562 28138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:35.777590 28138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:35.783542 28138 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:45155
I20260808 17:05:35.783578 28342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:45155 every 8 connection(s)
I20260808 17:05:35.783877 28138 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:05:35.786958 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 28138
I20260808 17:05:35.787071 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:05:35.787250 31496 external_mini_cluster.cc:1428] Setting key 92d439ecb4c4fe9f0ccb77b8c4221b7a
I20260808 17:05:35.788223 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40247
--builtin_ntp_servers=127.30.194.20:34557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:05:35.788913 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.784561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:35407 (local address 127.30.194.62:40247)
0808 17:05:35.784614 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:35.784616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:35.784822 (+ 206us) server_negotiation.cc:407] Connection header received
0808 17:05:35.785316 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:35.785318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:35.785328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:35.785381 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:35.786011 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.786492 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.787669 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.787891 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.787950 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:35.788669 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:35.788671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:35.788671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:35.788675 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:35.788692 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:35.788696 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:35.788754 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:35.788806 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 17:05:35.788837 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:05:35.789047 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.784536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40247 (local address 127.30.194.1:35407)
0808 17:05:35.784703 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:35.784734 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:35.785208 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:35.785432 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:35.785435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:35.785444 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:35.785948 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:35.785952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.786553 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.786553 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:35.787598 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:35.787602 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.787970 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.787971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:35.788083 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:35.788544 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:05:35.788545 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:35.788571 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:05:35.788592 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:35.788754 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:35.788756 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:35.788759 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:35.788807 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 17:05:35.788877 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260808 17:05:35.789580 28343 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40247
I20260808 17:05:35.789659 28343 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:35.789835 28343 heartbeater.cc:511] Master 127.30.194.62:40247 requested a full tablet report, sending...
I20260808 17:05:35.790385 28041 ts_manager.cc:194] Registered new tserver with Master: 9604baf931b242e58346c626009357ab (127.30.194.1:45155)
I20260808 17:05:35.791211 28041 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.1:35407
W20260808 17:05:35.869002 28349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:35.869215 28349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:35.869239 28349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:35.870465 28349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:35.870507 28349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:35.870530 28349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:05:35.871800 28349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:35.871999 28349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:35.872258 28349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:35.876842 28361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:35.876928 28349 server_base.cc:1034] running on GCE node
W20260808 17:05:35.876883 28365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:35.876899 28358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:35.877224 28349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:35.877456 28349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:35.878587 28349 hybrid_clock.cc:648] HybridClock initialized: now 1786208735878577 us; error 26 us; skew 500 ppm
I20260808 17:05:35.879660 28349 webserver.cc:460] Webserver started at http://127.30.194.2:37783/ using document root <none> and password file <none>
I20260808 17:05:35.879873 28349 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:35.879920 28349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:35.880044 28349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:35.880992 28349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "6b348e263f894a77945335e351475934"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "831125a2758ee87227532d8cd436c286"
server_key_iv: "55a59a3b5207924b29e2a213832755cb"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.881354 28349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "6b348e263f894a77945335e351475934"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "831125a2758ee87227532d8cd436c286"
server_key_iv: "55a59a3b5207924b29e2a213832755cb"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.882498 28349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:05:35.883121 28374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:35.883265 28349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:35.883342 28349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "6b348e263f894a77945335e351475934"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "831125a2758ee87227532d8cd436c286"
server_key_iv: "55a59a3b5207924b29e2a213832755cb"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.883407 28349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:35.896474 28349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:35.896729 28349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:35.896848 28349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:35.897035 28349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:35.897351 28349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:35.897384 28349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:35.897411 28349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:35.897428 28349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:35.902673 28349 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:41339
I20260808 17:05:35.902755 28549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:41339 every 8 connection(s)
I20260808 17:05:35.903014 28349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:05:35.903820 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 28349
I20260808 17:05:35.903928 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:05:35.904109 31496 external_mini_cluster.cc:1428] Setting key a93b0f885fa4c2580d7907a6fe1ce8ac
I20260808 17:05:35.904959 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40247
--builtin_ntp_servers=127.30.194.20:34557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:05:35.907970 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.903673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:48353 (local address 127.30.194.62:40247)
0808 17:05:35.903731 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:35.903734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:35.903948 (+ 214us) server_negotiation.cc:407] Connection header received
0808 17:05:35.904493 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:35.904494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:35.904506 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:35.904548 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:35.905280 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.905785 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.906721 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.906914 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.906969 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:35.907702 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:35.907704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:35.907704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:35.907709 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:35.907726 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:35.907729 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:35.907774 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:35.907850 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 17:05:35.907893 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 17:05:35.908149 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:35.903660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40247 (local address 127.30.194.2:48353)
0808 17:05:35.903843 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:35.903874 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:35.904408 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:35.904611 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:35.904615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:35.904625 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:35.905190 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:35.905196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.905866 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.905867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:35.906636 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:35.906638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:35.906981 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:35.906982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:35.907118 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:35.907574 (+ 456us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:05:35.907574 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:35.907604 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:05:35.907624 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:35.907781 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:35.907786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:35.907790 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:05:35.907851 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 17:05:35.907948 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260808 17:05:35.908548 28552 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40247
I20260808 17:05:35.908627 28552 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:35.908785 28552 heartbeater.cc:511] Master 127.30.194.62:40247 requested a full tablet report, sending...
I20260808 17:05:35.909147 28041 ts_manager.cc:194] Registered new tserver with Master: 6b348e263f894a77945335e351475934 (127.30.194.2:41339)
I20260808 17:05:35.909619 28041 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.2:48353
W20260808 17:05:35.984601 28558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:35.984773 28558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:35.984793 28558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:35.985890 28558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:35.985926 28558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:35.985963 28558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:05:35.987316 28558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:34557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:35.987493 28558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:35.987715 28558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:35.992396 28564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:35.992394 28567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:35.992375 28569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:35.992601 28558 server_base.cc:1034] running on GCE node
I20260808 17:05:35.992805 28558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:35.993029 28558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:35.994167 28558 hybrid_clock.cc:648] HybridClock initialized: now 1786208735994170 us; error 47 us; skew 500 ppm
I20260808 17:05:35.995378 28558 webserver.cc:460] Webserver started at http://127.30.194.3:45281/ using document root <none> and password file <none>
I20260808 17:05:35.995600 28558 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:35.995647 28558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:35.995779 28558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:35.996738 28558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "016f6bcb51f147449dcd044f29d97fab"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "c4ce81027de52f650c2b8370e62bc832"
server_key_iv: "5e2106b81794c19cba37f5c74e7a4f1e"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.997052 28558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "016f6bcb51f147449dcd044f29d97fab"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "c4ce81027de52f650c2b8370e62bc832"
server_key_iv: "5e2106b81794c19cba37f5c74e7a4f1e"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.998207 28558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:05:35.998793 28582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:35.998936 28558 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:35.999017 28558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "016f6bcb51f147449dcd044f29d97fab"
format_stamp: "Formatted at 2026-08-08 17:05:35 on dist-test-slave-2j7r"
server_key: "c4ce81027de52f650c2b8370e62bc832"
server_key_iv: "5e2106b81794c19cba37f5c74e7a4f1e"
server_key_version: "encryptionkey@0"
I20260808 17:05:35.999073 28558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:36.013334 28558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:36.013576 28558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:36.013695 28558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:36.013890 28558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:36.014187 28558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:36.014221 28558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:36.014249 28558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:36.014267 28558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:36.019644 28558 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:43083
I20260808 17:05:36.019680 28774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:43083 every 8 connection(s)
I20260808 17:05:36.019971 28558 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:05:36.020459 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 28558
I20260808 17:05:36.020569 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:05:36.020762 31496 external_mini_cluster.cc:1428] Setting key eee4ab2857cf054f2601a95acc01e218
I20260808 17:05:36.024855 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.020626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:46963 (local address 127.30.194.62:40247)
0808 17:05:36.020681 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:36.020684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:36.020954 (+ 270us) server_negotiation.cc:407] Connection header received
0808 17:05:36.021522 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:36.021523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:36.021537 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:36.021584 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:36.022286 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.022780 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.023695 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.023895 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.023948 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.024609 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:36.024611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:36.024611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:36.024615 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:36.024630 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:36.024633 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:36.024677 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:36.024752 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 17:05:36.024776 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:05:36.025066 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.020610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40247 (local address 127.30.194.3:46963)
0808 17:05:36.020821 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:36.020876 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:36.021430 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:36.021661 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:36.021666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:36.021674 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:36.022209 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.022214 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.022864 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.022865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.023617 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.023620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.023962 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.023963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.024092 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.024492 (+ 400us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:05:36.024492 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:36.024517 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:05:36.024537 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:36.024683 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:36.024688 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:36.024691 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:36.024750 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:05:36.024878 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260808 17:05:36.025547 28777 heartbeater.cc:347] Connected to a master server at 127.30.194.62:40247
I20260808 17:05:36.025616 28777 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:36.025748 28777 heartbeater.cc:511] Master 127.30.194.62:40247 requested a full tablet report, sending...
I20260808 17:05:36.026070 28041 ts_manager.cc:194] Registered new tserver with Master: 016f6bcb51f147449dcd044f29d97fab (127.30.194.3:43083)
I20260808 17:05:36.026417 28041 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.3:46963
I20260808 17:05:36.032277 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:05:36.035823 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.033519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.30.194.62:40247)
0808 17:05:36.033576 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:36.033578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:36.033642 (+ 64us) server_negotiation.cc:407] Connection header received
0808 17:05:36.033679 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:36.033680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:36.033687 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:36.033728 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:36.033846 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.034271 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.034960 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.035125 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.035164 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.035669 (+ 505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:36.035669 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:36.035670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:36.035672 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:36.035681 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:36.035684 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:36.035711 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:36.035754 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 17:05:36.035767 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:05:36.037050 28041 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:05:36.038523 28041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:05:36.044242 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.040460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:45155 (local address 127.0.0.1:57824)
0808 17:05:36.040715 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:36.040752 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:36.040821 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:36.041028 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:36.041031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:36.041033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:36.041109 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.041111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.042065 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.042065 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.043211 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.043214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.043962 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.043963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.044036 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.044049 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:05:36.044113 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:05:36.044165 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":48,"threads_started":1}
I20260808 17:05:36.044345 28796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.040646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:41339 (local address 127.0.0.1:34066)
0808 17:05:36.040929 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:36.040937 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:36.040974 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:36.041248 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:36.041249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:36.041251 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:36.041276 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.041277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.042527 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.042528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.043287 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.043290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.044104 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.044105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.044167 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.044172 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:05:36.044201 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 17:05:36.044239 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":7,"thread_start_us":69,"threads_started":1}
I20260808 17:05:36.044598 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.040570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57824 (local address 127.30.194.1:45155)
0808 17:05:36.040749 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:36.040753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:36.040815 (+ 62us) server_negotiation.cc:407] Connection header received
0808 17:05:36.040899 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:36.040902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:36.040919 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:36.040990 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:36.041174 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.041990 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.043306 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.043893 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.043952 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.044040 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:36.044485 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 17:05:36.044528 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260808 17:05:36.044623 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.040472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:43083 (local address 127.0.0.1:57526)
0808 17:05:36.040878 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:36.040888 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:36.040948 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:36.041207 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:36.041208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:36.041210 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:36.041244 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.041245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.042468 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.042469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.043518 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.043521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.044421 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.044422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.044481 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.044484 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:36.044531 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:05:36.044564 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":94,"threads_started":1}
I20260808 17:05:36.044636 28798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.040747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.30.194.2:41339)
0808 17:05:36.041113 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:36.041118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:36.041127 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:05:36.041163 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:36.041165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:36.041179 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:36.041252 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:36.041370 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.042437 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.043371 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.044004 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.044411 (+ 407us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.044495 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:36.044519 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 17:05:36.044553 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":94,"threads_started":1}
I20260808 17:05:36.045290 28444 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce580c604b342f28f75825805096910 (DEFAULT_TABLE table=test-table [id=8f88847610ea4814bf535ad4b337d2a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:36.045290 28223 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce580c604b342f28f75825805096910 (DEFAULT_TABLE table=test-table [id=8f88847610ea4814bf535ad4b337d2a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:36.045641 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.040558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57526 (local address 127.30.194.3:43083)
0808 17:05:36.040824 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:36.040827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:36.040939 (+ 112us) server_negotiation.cc:407] Connection header received
0808 17:05:36.041029 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:36.041031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:36.041046 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:36.041156 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:36.041325 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.042381 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.043594 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.044240 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.044306 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.044387 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:36.044765 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 17:05:36.044803 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":75,"threads_started":1}
I20260808 17:05:36.045652 28223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce580c604b342f28f75825805096910. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:36.045652 28444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce580c604b342f28f75825805096910. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:36.046151 28660 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce580c604b342f28f75825805096910 (DEFAULT_TABLE table=test-table [id=8f88847610ea4814bf535ad4b337d2a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:36.046451 28660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce580c604b342f28f75825805096910. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:36.048771 28799 tablet_bootstrap.cc:492] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: Bootstrap starting.
I20260808 17:05:36.049033 28802 tablet_bootstrap.cc:492] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab: Bootstrap starting.
I20260808 17:05:36.049220 28800 tablet_bootstrap.cc:492] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: Bootstrap starting.
I20260808 17:05:36.049487 28799 tablet_bootstrap.cc:654] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:36.049759 28799 log.cc:824] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:36.049794 28800 tablet_bootstrap.cc:654] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:36.050079 28800 log.cc:824] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:36.050410 28802 tablet_bootstrap.cc:654] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:36.050444 28799 tablet_bootstrap.cc:492] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: No bootstrap required, opened a new log
I20260808 17:05:36.050505 28799 ts_tablet_manager.cc:1397] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:05:36.050681 28802 log.cc:824] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:36.050750 28800 tablet_bootstrap.cc:492] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: No bootstrap required, opened a new log
I20260808 17:05:36.050813 28800 ts_tablet_manager.cc:1397] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:05:36.051286 28802 tablet_bootstrap.cc:492] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab: No bootstrap required, opened a new log
I20260808 17:05:36.051342 28802 ts_tablet_manager.cc:1397] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:05:36.053166 28799 raft_consensus.cc:348] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 } } peers { permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 } } peers { permanent_uuid: "6b348e263f894a77945335e351475934" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41339 } }
I20260808 17:05:36.053166 28800 raft_consensus.cc:348] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 } } peers { permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 } } peers { permanent_uuid: "6b348e263f894a77945335e351475934" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41339 } }
I20260808 17:05:36.053213 28802 raft_consensus.cc:348] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 } } peers { permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 } } peers { permanent_uuid: "6b348e263f894a77945335e351475934" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41339 } }
I20260808 17:05:36.053328 28802 raft_consensus.cc:374] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:36.053329 28799 raft_consensus.cc:374] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:36.053329 28800 raft_consensus.cc:374] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:36.053359 28802 raft_consensus.cc:729] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9604baf931b242e58346c626009357ab, State: Initialized, Role: FOLLOWER
I20260808 17:05:36.053367 28800 raft_consensus.cc:729] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b348e263f894a77945335e351475934, State: Initialized, Role: FOLLOWER
I20260808 17:05:36.053367 28799 raft_consensus.cc:729] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016f6bcb51f147449dcd044f29d97fab, State: Initialized, Role: FOLLOWER
I20260808 17:05:36.053467 28802 consensus_queue.cc:260] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 } } peers { permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 } } peers { permanent_uuid: "6b348e263f894a77945335e351475934" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41339 } }
I20260808 17:05:36.053471 28800 consensus_queue.cc:260] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 } } peers { permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 } } peers { permanent_uuid: "6b348e263f894a77945335e351475934" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41339 } }
I20260808 17:05:36.053471 28799 consensus_queue.cc:260] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 } } peers { permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 } } peers { permanent_uuid: "6b348e263f894a77945335e351475934" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41339 } }
I20260808 17:05:36.053723 28800 ts_tablet_manager.cc:1428] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:05:36.053723 28802 ts_tablet_manager.cc:1428] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 17:05:36.053776 28343 heartbeater.cc:503] Master 127.30.194.62:40247 was elected leader, sending a full tablet report...
I20260808 17:05:36.053922 28552 heartbeater.cc:503] Master 127.30.194.62:40247 was elected leader, sending a full tablet report...
I20260808 17:05:36.054023 28799 ts_tablet_manager.cc:1428] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:05:36.054128 28777 heartbeater.cc:503] Master 127.30.194.62:40247 was elected leader, sending a full tablet report...
W20260808 17:05:36.153529 28553 tablet.cc:2369] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:05:36.270552 28779 tablet.cc:2369] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:05:36.284547 28346 tablet.cc:2369] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:05:36.437071 28809 raft_consensus.cc:482] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:05:36.437218 28809 raft_consensus.cc:504] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 } } peers { permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 } } peers { permanent_uuid: "6b348e263f894a77945335e351475934" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41339 } }
I20260808 17:05:36.437578 28809 leader_election.cc:288] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9604baf931b242e58346c626009357ab (127.30.194.1:45155), 016f6bcb51f147449dcd044f29d97fab (127.30.194.3:43083)
I20260808 17:05:36.441439 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.437840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:60233 (local address 127.30.194.3:43083)
0808 17:05:36.437896 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:36.437899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:36.437987 (+ 88us) server_negotiation.cc:407] Connection header received
0808 17:05:36.438053 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:36.438054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:36.438064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:36.438141 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:36.438324 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.439060 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.440534 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.441076 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.441171 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.441180 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:36.441327 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 17:05:36.441362 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:05:36.441439 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.437900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:52945 (local address 127.30.194.1:45155)
0808 17:05:36.437938 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:36.437940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:36.437981 (+ 41us) server_negotiation.cc:407] Connection header received
0808 17:05:36.438053 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:36.438054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:36.438064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:36.438134 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:36.438320 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.439060 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.440539 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.441080 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.441171 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.441180 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:36.441326 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 17:05:36.441363 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:05:36.441481 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.437761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:45155 (local address 127.30.194.2:52945)
0808 17:05:36.437914 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:36.437926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:36.437992 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:36.438153 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:36.438154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:36.438157 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:36.438256 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.438257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.439146 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.439148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.440443 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.440446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.441185 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.441186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.441260 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.441263 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:36.441313 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:05:36.441360 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":60,"thread_start_us":89,"threads_started":1}
I20260808 17:05:36.441481 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.437761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:43083 (local address 127.30.194.2:60233)
0808 17:05:36.437914 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:36.437926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:36.437992 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:36.438153 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:36.438154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:36.438156 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:36.438253 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.438255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.439146 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.439148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.440443 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:36.440446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.441174 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.441175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:36.441253 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.441258 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:05:36.441313 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:05:36.441362 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":90,"thread_start_us":76,"threads_started":1}
I20260808 17:05:36.441847 28267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce580c604b342f28f75825805096910" candidate_uuid: "6b348e263f894a77945335e351475934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9604baf931b242e58346c626009357ab" is_pre_election: true
I20260808 17:05:36.442003 28267 raft_consensus.cc:2454] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b348e263f894a77945335e351475934 in term 0.
I20260808 17:05:36.442121 28697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce580c604b342f28f75825805096910" candidate_uuid: "6b348e263f894a77945335e351475934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016f6bcb51f147449dcd044f29d97fab" is_pre_election: true
I20260808 17:05:36.442237 28378 leader_election.cc:302] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b348e263f894a77945335e351475934, 9604baf931b242e58346c626009357ab; no voters:
I20260808 17:05:36.442284 28697 raft_consensus.cc:2454] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b348e263f894a77945335e351475934 in term 0.
I20260808 17:05:36.442442 28809 raft_consensus.cc:2781] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:05:36.442498 28809 raft_consensus.cc:482] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:05:36.442514 28809 raft_consensus.cc:3037] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:36.443228 28809 raft_consensus.cc:504] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 } } peers { permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 } } peers { permanent_uuid: "6b348e263f894a77945335e351475934" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41339 } }
I20260808 17:05:36.443367 28809 leader_election.cc:288] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [CANDIDATE]: Term 1 election: Requested vote from peers 9604baf931b242e58346c626009357ab (127.30.194.1:45155), 016f6bcb51f147449dcd044f29d97fab (127.30.194.3:43083)
I20260808 17:05:36.443501 28267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce580c604b342f28f75825805096910" candidate_uuid: "6b348e263f894a77945335e351475934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9604baf931b242e58346c626009357ab"
I20260808 17:05:36.443516 28697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce580c604b342f28f75825805096910" candidate_uuid: "6b348e263f894a77945335e351475934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016f6bcb51f147449dcd044f29d97fab"
I20260808 17:05:36.443575 28697 raft_consensus.cc:3037] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:36.443579 28267 raft_consensus.cc:3037] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:36.444300 28697 raft_consensus.cc:2454] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b348e263f894a77945335e351475934 in term 1.
I20260808 17:05:36.444334 28267 raft_consensus.cc:2454] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b348e263f894a77945335e351475934 in term 1.
I20260808 17:05:36.444468 28380 leader_election.cc:302] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016f6bcb51f147449dcd044f29d97fab, 6b348e263f894a77945335e351475934; no voters:
I20260808 17:05:36.444582 28809 raft_consensus.cc:2781] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:36.444739 28809 raft_consensus.cc:686] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 1 LEADER]: Becoming Leader. State: Replica: 6b348e263f894a77945335e351475934, State: Running, Role: LEADER
I20260808 17:05:36.444840 28809 consensus_queue.cc:237] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 } } peers { permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 } } peers { permanent_uuid: "6b348e263f894a77945335e351475934" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41339 } }
I20260808 17:05:36.445688 28039 catalog_manager.cc:5495] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b348e263f894a77945335e351475934 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "6b348e263f894a77945335e351475934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b348e263f894a77945335e351475934" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 41339 } health_report { overall_health: HEALTHY } } }
I20260808 17:05:36.482379 28798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.480368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34068 (local address 127.30.194.2:41339)
0808 17:05:36.480434 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:36.480436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:36.480444 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:36.480483 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:36.480484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:36.480502 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:36.480559 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:05:36.480653 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.481120 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.481817 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:36.481998 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:36.482038 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:36.482133 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:05:36.482228 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:05:36.482279 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:36.482289 (+ 10us) server_negotiation.cc:299] Negotiation successful
0808 17:05:36.482304 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 17:05:36.484072 28697 raft_consensus.cc:1261] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab [term 1 FOLLOWER]: Refusing update from remote peer 6b348e263f894a77945335e351475934: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:36.484097 28267 raft_consensus.cc:1261] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab [term 1 FOLLOWER]: Refusing update from remote peer 6b348e263f894a77945335e351475934: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:36.484418 28818 consensus_queue.cc:1035] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016f6bcb51f147449dcd044f29d97fab" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 43083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:36.484540 28809 consensus_queue.cc:1035] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9604baf931b242e58346c626009357ab" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 45155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:36.488196 28823 mvcc.cc:205] Tried to move back new op lower bound from 7316310984636325888 to 7316310984478695424. Current Snapshot: MvccSnapshot[applied={T|T < 7316310984636325888 or (T in {7316310984636325888})}]
I20260808 17:05:36.488097 28825 mvcc.cc:205] Tried to move back new op lower bound from 7316310984636325888 to 7316310984478695424. Current Snapshot: MvccSnapshot[applied={T|T < 7316310984636325888 or (T in {7316310984636325888})}]
I20260808 17:05:36.489212 28041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:36.489297 28041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:36.489961 28041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa828a980aa847379166c5e554253ab9: Sending DeleteTablet for 3 replicas of tablet 8ce580c604b342f28f75825805096910
I20260808 17:05:36.490259 28223 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ce580c604b342f28f75825805096910 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:36 UTC) from {username='slave'} at 127.0.0.1:57824
I20260808 17:05:36.490288 28444 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ce580c604b342f28f75825805096910 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:36 UTC) from {username='slave'} at 127.0.0.1:34066
I20260808 17:05:36.490319 28660 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ce580c604b342f28f75825805096910 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:36 UTC) from {username='slave'} at 127.0.0.1:57526
I20260808 17:05:36.490506 28840 tablet_replica.cc:330] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab: stopping tablet replica
I20260808 17:05:36.490556 28843 tablet_replica.cc:330] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: stopping tablet replica
I20260808 17:05:36.490604 28840 raft_consensus.cc:2229] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:36.490644 28843 raft_consensus.cc:2229] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:36.490711 28840 raft_consensus.cc:2258] T 8ce580c604b342f28f75825805096910 P 9604baf931b242e58346c626009357ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:36.490789 28843 raft_consensus.cc:2258] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:36.490871 28842 tablet_replica.cc:330] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: stopping tablet replica
I20260808 17:05:36.490901 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 28138
I20260808 17:05:36.490957 28842 raft_consensus.cc:2229] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:05:36.491068 28842 raft_consensus.cc:2258] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:36.491443 28842 ts_tablet_manager.cc:1905] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:36.493060 28842 ts_tablet_manager.cc:1918] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:36.493878 28842 log.cc:1196] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/8ce580c604b342f28f75825805096910
I20260808 17:05:36.494203 28842 ts_tablet_manager.cc:1939] T 8ce580c604b342f28f75825805096910 P 6b348e263f894a77945335e351475934: Deleting consensus metadata
I20260808 17:05:36.495893 28014 catalog_manager.cc:4841] TS 6b348e263f894a77945335e351475934 (127.30.194.2:41339): tablet 8ce580c604b342f28f75825805096910 (table test-table [id=8f88847610ea4814bf535ad4b337d2a9]) successfully deleted
I20260808 17:05:36.498167 28843 ts_tablet_manager.cc:1905] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:36.499887 28843 ts_tablet_manager.cc:1918] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:36.499958 28843 log.cc:1196] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/8ce580c604b342f28f75825805096910
I20260808 17:05:36.500214 28843 ts_tablet_manager.cc:1939] T 8ce580c604b342f28f75825805096910 P 016f6bcb51f147449dcd044f29d97fab: Deleting consensus metadata
I20260808 17:05:36.500620 28020 catalog_manager.cc:4841] TS 016f6bcb51f147449dcd044f29d97fab (127.30.194.3:43083): tablet 8ce580c604b342f28f75825805096910 (table test-table [id=8f88847610ea4814bf535ad4b337d2a9]) successfully deleted
W20260808 17:05:36.507872 28014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:45155 (error 108)
I20260808 17:05:36.508404 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:36.508264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:45155 (local address 127.0.0.1:57826)
0808 17:05:36.508326 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:36.508371 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:45155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:05:36.508515 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 28349
W20260808 17:05:36.514086 28014 catalog_manager.cc:4584] TS 9604baf931b242e58346c626009357ab (127.30.194.1:45155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ce580c604b342f28f75825805096910: Network error: Client connection negotiation failed: client connection to 127.30.194.1:45155: connect: Connection refused (error 111)
I20260808 17:05:36.517047 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 28558
I20260808 17:05:36.534040 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 27989
2026-08-08T17:05:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1071 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:36 dist-test-slave-2j7r krb5kdc[28848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:36 dist-test-slave-2j7r krb5kdc[28848](info): set up 2 sockets
Aug 08 17:05:36 dist-test-slave-2j7r krb5kdc[28848](info): commencing operation
krb5kdc: starting...
W20260808 17:05:37.690486 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:37 dist-test-slave-2j7r krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:05:37.799506 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:36813
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:36813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:37.881752 28865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:37.881935 28865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:37.881965 28865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:37.883095 28865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:37.883136 28865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:37.883154 28865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:37.883170 28865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:37.883186 28865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:37.884646 28865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:36813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:36813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:37.884868 28865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:37.885100 28865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:37.889612 28878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:37.889633 28874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:37.889611 28876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:37.889775 28865 server_base.cc:1034] running on GCE node
I20260808 17:05:37.890141 28865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:37.890417 28865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:37.891577 28865 hybrid_clock.cc:648] HybridClock initialized: now 1786208737891560 us; error 28 us; skew 500 ppm
Aug 08 17:05:37 dist-test-slave-2j7r krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208737, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:37.893829 28865 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:05:37.894233 28865 webserver.cc:460] Webserver started at http://127.30.194.62:43433/ using document root <none> and password file <none>
I20260808 17:05:37.894448 28865 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:37.894495 28865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:37.894615 28865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:37.895560 28865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "3ebbdc9208b34b16998373930917c49b"
format_stamp: "Formatted at 2026-08-08 17:05:37 on dist-test-slave-2j7r"
server_key: "d9e9cacef256fa0dbdcae59cdbc0a9c6"
server_key_iv: "a93d3efa14b7da8ada4d0c670b9d981e"
server_key_version: "encryptionkey@0"
I20260808 17:05:37.895872 28865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "3ebbdc9208b34b16998373930917c49b"
format_stamp: "Formatted at 2026-08-08 17:05:37 on dist-test-slave-2j7r"
server_key: "d9e9cacef256fa0dbdcae59cdbc0a9c6"
server_key_iv: "a93d3efa14b7da8ada4d0c670b9d981e"
server_key_version: "encryptionkey@0"
I20260808 17:05:37.897058 28865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:05:37.897789 28893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:37.897933 28865 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:37.898013 28865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "3ebbdc9208b34b16998373930917c49b"
format_stamp: "Formatted at 2026-08-08 17:05:37 on dist-test-slave-2j7r"
server_key: "d9e9cacef256fa0dbdcae59cdbc0a9c6"
server_key_iv: "a93d3efa14b7da8ada4d0c670b9d981e"
server_key_version: "encryptionkey@0"
I20260808 17:05:37.898072 28865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:37.906575 28865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:37.908025 28865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:37.908140 28865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:37.911966 28865 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:36813
I20260808 17:05:37.912020 28977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:36813 every 8 connection(s)
I20260808 17:05:37.912307 28865 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:37.913067 28982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:37.913372 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 28865
I20260808 17:05:37.913475 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:37.913662 31496 external_mini_cluster.cc:1428] Setting key f3c3e0e4d87cd02797e0cfb6f1ea83ec
I20260808 17:05:37.915786 28982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b: Bootstrap starting.
I20260808 17:05:37.916328 28982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:37.916605 28982 log.cc:824] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:37.917323 28982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b: No bootstrap required, opened a new log
I20260808 17:05:37.919519 28982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ebbdc9208b34b16998373930917c49b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36813 } }
I20260808 17:05:37.919649 28982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:37.919672 28982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ebbdc9208b34b16998373930917c49b, State: Initialized, Role: FOLLOWER
Aug 08 17:05:37 dist-test-slave-2j7r krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:37.919785 28982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ebbdc9208b34b16998373930917c49b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36813 } }
I20260808 17:05:37.919842 28982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:37.919876 28982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:37.919921 28982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:37.920485 28982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ebbdc9208b34b16998373930917c49b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36813 } }
I20260808 17:05:37.920589 28982 leader_election.cc:302] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ebbdc9208b34b16998373930917c49b; no voters:
I20260808 17:05:37.920781 28982 leader_election.cc:288] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:37.920822 28990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:37.920964 28990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [term 1 LEADER]: Becoming Leader. State: Replica: 3ebbdc9208b34b16998373930917c49b, State: Running, Role: LEADER
I20260808 17:05:37.921025 28982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:37.921119 28990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ebbdc9208b34b16998373930917c49b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36813 } }
I20260808 17:05:37.922351 28994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ebbdc9208b34b16998373930917c49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ebbdc9208b34b16998373930917c49b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36813 } } }
I20260808 17:05:37.922432 28994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:37.922827 28992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ebbdc9208b34b16998373930917c49b. Latest consensus state: current_term: 1 leader_uuid: "3ebbdc9208b34b16998373930917c49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ebbdc9208b34b16998373930917c49b" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 36813 } } }
I20260808 17:05:37.922964 28992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b [sys.catalog]: This master's current role is: LEADER
W20260808 17:05:37.923563 29012 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:05:37.923647 29012 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:05:37.923717 29014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:37.923877 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:37.914127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42166 (local address 127.30.194.62:36813)
0808 17:05:37.914281 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:37.914288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:37.914310 (+ 22us) server_negotiation.cc:407] Connection header received
0808 17:05:37.914695 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:37.914700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:37.914743 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:37.914851 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:37.915560 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:37.916442 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:37.917396 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:37.917784 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:37.917865 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:37.920276 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:37.920281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:37.920281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:37.920293 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:37.922169 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:37.922595 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:37.922598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:37.922598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:37.922698 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:37.922955 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:37.922956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:37.922956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:37.923254 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:37.923394 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:37.923524 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 17:05:37.923652 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260808 17:05:37.923884 29014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:37.925276 29014 catalog_manager.cc:1347] Generated new cluster ID: 00ca99c2a3bf49da8dd479c3e7a1c972
I20260808 17:05:37.925328 29014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:37.945364 29014 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:37.945851 29014 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:37.953056 29014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b: Generated new TSK 0
I20260808 17:05:37.953234 29014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:05:37.983657 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36813
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:38.074497 29020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:38.074676 29020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:38.074697 29020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:38.075773 29020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:38.075812 29020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:38.075839 29020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:05:38.077270 29020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:38.077471 29020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:38.077667 29020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:38.082196 29033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:38.082188 29029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:38.082376 29020 server_base.cc:1034] running on GCE node
W20260808 17:05:38.082196 29028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:38.082588 29020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:38.082810 29020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:38.083947 29020 hybrid_clock.cc:648] HybridClock initialized: now 1786208738083935 us; error 27 us; skew 500 ppm
Aug 08 17:05:38 dist-test-slave-2j7r krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:38.086283 29020 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:05:38.086678 29020 webserver.cc:460] Webserver started at http://127.30.194.1:37833/ using document root <none> and password file <none>
I20260808 17:05:38.086866 29020 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:38.086910 29020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:38.087018 29020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:38.087934 29020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "408d222821964c57a083348d80240c8f"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "0dd8a844acc5620c8019ff7d9ec638ec"
server_key_iv: "d250dc38894c9dbf8cd1f620fa14df14"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.088244 29020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "408d222821964c57a083348d80240c8f"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "0dd8a844acc5620c8019ff7d9ec638ec"
server_key_iv: "d250dc38894c9dbf8cd1f620fa14df14"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.089526 29020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 17:05:38.090341 29045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.090519 29020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:05:38.090601 29020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "408d222821964c57a083348d80240c8f"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "0dd8a844acc5620c8019ff7d9ec638ec"
server_key_iv: "d250dc38894c9dbf8cd1f620fa14df14"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.090667 29020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:38.102644 29020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:38.104169 29020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:38.104293 29020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:38.104472 29020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:38.104732 29020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:38.104764 29020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.104789 29020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:38.104813 29020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.110208 29020 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:37925
I20260808 17:05:38.110245 29231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:37925 every 8 connection(s)
I20260808 17:05:38.110505 29020 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:05:38 dist-test-slave-2j7r krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:38.118527 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 29020
I20260808 17:05:38.118566 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.111186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:44771 (local address 127.30.194.62:36813)
0808 17:05:38.111261 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.111263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.111408 (+ 145us) server_negotiation.cc:407] Connection header received
0808 17:05:38.111854 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.111855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.111867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.111919 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:38.112835 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.113354 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.114290 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.114486 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.114541 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.116411 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:38.116412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:38.116413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:38.116420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:38.117511 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:38.117834 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:38.117834 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:38.117835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:38.117881 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:38.118045 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:38.118046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:38.118046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:38.118133 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:38.118175 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.118383 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.118478 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:05:38.118619 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.111110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36813 (local address 127.30.194.1:44771)
0808 17:05:38.111293 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:38.111333 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:38.111759 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:38.111989 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:38.111994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:38.112568 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:38.112743 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.112747 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.113441 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.113443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.114198 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.114200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.114554 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.114555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.114670 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.115069 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:38.115077 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:38.116313 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:38.117581 (+ 1268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:38.117587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:38.117589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:38.117782 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:38.117924 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:38.117925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:38.117925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:38.118006 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:38.118172 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:38.118175 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:38.118270 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 17:05:38.118361 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 17:05:38.118469 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260808 17:05:38.118638 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:05:38.118825 31496 external_mini_cluster.cc:1428] Setting key 27f2826e86ef4826aa33d557b4ec12c6
I20260808 17:05:38.119083 29232 heartbeater.cc:347] Connected to a master server at 127.30.194.62:36813
I20260808 17:05:38.119169 29232 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:38.119329 29232 heartbeater.cc:511] Master 127.30.194.62:36813 requested a full tablet report, sending...
I20260808 17:05:38.119822 28919 ts_manager.cc:194] Registered new tserver with Master: 408d222821964c57a083348d80240c8f (127.30.194.1:37925)
I20260808 17:05:38.120550 28919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:44771
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:05:38.141412 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36813
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:38.221586 29240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:38.221812 29240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:38.221849 29240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:38.223008 29240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:38.223055 29240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:38.223093 29240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:05:38.224521 29240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:38.224743 29240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:38.224944 29240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:38.229738 29250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:38.229770 29249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:38.229795 29253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:38.229939 29240 server_base.cc:1034] running on GCE node
I20260808 17:05:38.230180 29240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:38.230419 29240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:38.231554 29240 hybrid_clock.cc:648] HybridClock initialized: now 1786208738231538 us; error 33 us; skew 500 ppm
Aug 08 17:05:38 dist-test-slave-2j7r krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:38.233858 29240 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:05:38.234238 29240 webserver.cc:460] Webserver started at http://127.30.194.2:40425/ using document root <none> and password file <none>
I20260808 17:05:38.234431 29240 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:38.234475 29240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:38.234596 29240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:38.235571 29240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "c80f7e0a8fcb4474b62964f3b937530d"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "607f0bffc4f44151c339b47da4df728b"
server_key_iv: "fd057b43ac2bd8a58d7cf2645d4a595c"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.235884 29240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "c80f7e0a8fcb4474b62964f3b937530d"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "607f0bffc4f44151c339b47da4df728b"
server_key_iv: "fd057b43ac2bd8a58d7cf2645d4a595c"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.237111 29240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:05:38.237795 29268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.237950 29240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:38.238032 29240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "c80f7e0a8fcb4474b62964f3b937530d"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "607f0bffc4f44151c339b47da4df728b"
server_key_iv: "fd057b43ac2bd8a58d7cf2645d4a595c"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.238090 29240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:38.254065 29240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:38.255985 29240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:38.256108 29240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:38.256292 29240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:38.256562 29240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:38.256594 29240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.256620 29240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:38.256637 29240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.262008 29240 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:38179
I20260808 17:05:38.262046 29455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:38179 every 8 connection(s)
I20260808 17:05:38.262305 29240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:05:38.265416 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 29240
I20260808 17:05:38.265511 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:05:38.265677 31496 external_mini_cluster.cc:1428] Setting key 4a5521d5eede6b7be9139e578ef558a1
Aug 08 17:05:38 dist-test-slave-2j7r krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:38.270581 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.262918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:43813 (local address 127.30.194.62:36813)
0808 17:05:38.262961 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.262963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.263141 (+ 178us) server_negotiation.cc:407] Connection header received
0808 17:05:38.263562 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.263563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.263571 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.263622 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:38.264449 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.264927 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.265950 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.266187 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.266245 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.268193 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:38.268195 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:38.268195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:38.268201 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:38.269363 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:38.269760 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:38.269761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:38.269761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:38.269804 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:38.270005 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:38.270005 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:38.270006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:38.270091 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:38.270133 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.270442 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.270519 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:05:38.270792 29461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.262887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36813 (local address 127.30.194.2:43813)
0808 17:05:38.263040 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:38.263078 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:38.263489 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:38.263686 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:38.263689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:38.264199 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:38.264384 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.264388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.264984 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.264985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.265844 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.265847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.266260 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.266261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.266394 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.266798 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:38.266806 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:38.268110 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:38.269432 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:38.269438 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:38.269441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:38.269699 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:38.269842 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:38.269842 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:38.269842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:38.269958 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:38.270144 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:38.270151 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:38.270316 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 17:05:38.270426 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 17:05:38.270583 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260808 17:05:38.271219 29458 heartbeater.cc:347] Connected to a master server at 127.30.194.62:36813
I20260808 17:05:38.271302 29458 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:38.271459 29458 heartbeater.cc:511] Master 127.30.194.62:36813 requested a full tablet report, sending...
I20260808 17:05:38.271812 28919 ts_manager.cc:194] Registered new tserver with Master: c80f7e0a8fcb4474b62964f3b937530d (127.30.194.2:38179)
I20260808 17:05:38.272297 28919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:43813
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:05:38.288805 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36813
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:41283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:38.369374 29464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:38.369560 29464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:38.369581 29464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:38.370656 29464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:38.370687 29464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:38.370710 29464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:05:38.372066 29464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:41283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:36813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:38.372254 29464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:38.372427 29464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:05:38.377069 29464 server_base.cc:1034] running on GCE node
W20260808 17:05:38.377049 29478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:38.377043 29474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:38.377063 29475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:38.377524 29464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:38.377733 29464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:38.378861 29464 hybrid_clock.cc:648] HybridClock initialized: now 1786208738378850 us; error 27 us; skew 500 ppm
Aug 08 17:05:38 dist-test-slave-2j7r krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:38.381172 29464 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:05:38.381565 29464 webserver.cc:460] Webserver started at http://127.30.194.3:45099/ using document root <none> and password file <none>
I20260808 17:05:38.381769 29464 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:38.381811 29464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:38.381929 29464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:38.382854 29464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "82e6464f7aa04a07aae25748764f42c2"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "c72d9650512432dc1ce9f4c29a736b26"
server_key_iv: "2285f920c0a2de9ee66c1c14a6c001f7"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.383150 29464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "82e6464f7aa04a07aae25748764f42c2"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "c72d9650512432dc1ce9f4c29a736b26"
server_key_iv: "2285f920c0a2de9ee66c1c14a6c001f7"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.384316 29464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:05:38.384935 29486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.385072 29464 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.385180 29464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "82e6464f7aa04a07aae25748764f42c2"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "c72d9650512432dc1ce9f4c29a736b26"
server_key_iv: "2285f920c0a2de9ee66c1c14a6c001f7"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.385242 29464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:38.429299 29464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:38.430923 29464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:38.431046 29464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:38.431241 29464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:38.431524 29464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:38.431555 29464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.431581 29464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:38.431605 29464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.437028 29464 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:45781
I20260808 17:05:38.437064 29674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:45781 every 8 connection(s)
I20260808 17:05:38.437379 29464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:05:38 dist-test-slave-2j7r krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:38.443370 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 29464
I20260808 17:05:38.443461 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:05:38.443655 31496 external_mini_cluster.cc:1428] Setting key ed07bc7a7b0e18f636c3dee8b059410c
I20260808 17:05:38.445551 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.437979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:44377 (local address 127.30.194.62:36813)
0808 17:05:38.438035 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.438037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.438236 (+ 199us) server_negotiation.cc:407] Connection header received
0808 17:05:38.438659 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.438660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.438669 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.438718 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:38.439540 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.440029 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.441022 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.441264 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.441314 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.443110 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:38.443112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:38.443112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:38.443119 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:38.444554 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:38.444869 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:38.444870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:38.444870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:38.444915 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:38.445064 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:38.445064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:38.445064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:38.445182 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:38.445247 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.445436 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.445492 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:05:38.445719 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.437953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:36813 (local address 127.30.194.3:44377)
0808 17:05:38.438098 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:38.438137 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:38.438587 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:38.438768 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:38.438772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:38.439293 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:38.439475 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.439480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.440103 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.440104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.440954 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.440957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.441319 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.441319 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.441418 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.441753 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:38.441761 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:38.443033 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:38.444611 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:38.444616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:38.444618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:38.444811 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:38.444947 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:38.444947 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:38.444948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:38.445027 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:38.445228 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:38.445231 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:38.445342 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 17:05:38.445416 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 17:05:38.445523 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260808 17:05:38.446136 29677 heartbeater.cc:347] Connected to a master server at 127.30.194.62:36813
I20260808 17:05:38.446214 29677 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:38.446367 29677 heartbeater.cc:511] Master 127.30.194.62:36813 requested a full tablet report, sending...
I20260808 17:05:38.446666 28919 ts_manager.cc:194] Registered new tserver with Master: 82e6464f7aa04a07aae25748764f42c2 (127.30.194.3:45781)
I20260808 17:05:38.447001 28919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:44377
I20260808 17:05:38.455030 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:05:38.461688 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.456249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42172 (local address 127.30.194.62:36813)
0808 17:05:38.456310 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.456312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.456404 (+ 92us) server_negotiation.cc:407] Connection header received
0808 17:05:38.456466 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.456467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.456476 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.456520 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:38.457001 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.457496 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.458172 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.458365 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.458416 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.459548 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:38.459549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:38.459550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:38.459556 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:38.460576 (+ 1020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:38.460928 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:38.460929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:38.460930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:38.460980 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:38.461184 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:38.461185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:38.461185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:38.461313 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:38.461422 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.461549 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.461605 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:05:38.462971 28919 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:05:38.464478 28919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:05:38.470806 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.466692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:45781 (local address 127.0.0.1:56574)
0808 17:05:38.467042 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:38.467078 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:38.467134 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:38.467441 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:38.467441 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:38.467443 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:38.467489 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.467490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.468744 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.468745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.469819 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.469821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.470482 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.470484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.470574 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.470588 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 17:05:38.470657 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 17:05:38.470721 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":73,"threads_started":1}
I20260808 17:05:38.470854 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.466864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47566 (local address 127.30.194.2:38179)
0808 17:05:38.467041 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.467046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.467152 (+ 106us) server_negotiation.cc:407] Connection header received
0808 17:05:38.467250 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.467253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.467269 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.467522 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:38.467660 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.468696 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.469837 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.470523 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.470591 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.470646 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.470746 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.470782 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
I20260808 17:05:38.471024 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.467049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49834 (local address 127.30.194.1:37925)
0808 17:05:38.467290 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.467295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.467306 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:05:38.467399 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.467402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.467419 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.467498 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:38.467729 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.468417 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.469906 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.470540 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.470593 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.470646 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.470911 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.470954 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":85,"threads_started":1}
I20260808 17:05:38.470866 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.466704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:38179 (local address 127.0.0.1:47566)
0808 17:05:38.467046 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:38.467078 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:38.467155 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:38.467385 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:38.467388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:38.467390 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:38.467447 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.467448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.468816 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.468817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.469759 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.469761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.470614 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.470615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.470670 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.470673 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:38.470732 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:05:38.470765 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260808 17:05:38.471349 29698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.466751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56574 (local address 127.30.194.3:45781)
0808 17:05:38.467255 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.467259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.467268 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:05:38.467299 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.467301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.467312 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.467376 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:38.467557 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.468640 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.469928 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.470398 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.470471 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.470530 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.471231 (+ 701us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.471274 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":66,"threads_started":1}
I20260808 17:05:38.470960 29695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.466909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37925 (local address 127.0.0.1:49834)
0808 17:05:38.467250 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:38.467259 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:38.467299 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:38.467581 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:38.467582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:38.467585 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:38.467645 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.467647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.468515 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.468516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.469817 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.469821 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.470680 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.470681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.470762 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.470767 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:05:38.470813 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 17:05:38.470850 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":29,"thread_start_us":69,"threads_started":1}
I20260808 17:05:38.471875 29347 tablet_service.cc:1467] Processing CreateTablet for tablet 11bc00273c914a52b14d8ba0dd96fcbd (DEFAULT_TABLE table=test-table [id=40be2f0060f44149bbdb0b6a762ea833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:38.471932 29559 tablet_service.cc:1467] Processing CreateTablet for tablet 11bc00273c914a52b14d8ba0dd96fcbd (DEFAULT_TABLE table=test-table [id=40be2f0060f44149bbdb0b6a762ea833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:38.472240 29559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11bc00273c914a52b14d8ba0dd96fcbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:38.472235 29347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11bc00273c914a52b14d8ba0dd96fcbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:38.472187 29124 tablet_service.cc:1467] Processing CreateTablet for tablet 11bc00273c914a52b14d8ba0dd96fcbd (DEFAULT_TABLE table=test-table [id=40be2f0060f44149bbdb0b6a762ea833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:38.472432 29124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11bc00273c914a52b14d8ba0dd96fcbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:38.474834 29702 tablet_bootstrap.cc:492] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f: Bootstrap starting.
I20260808 17:05:38.475392 29702 tablet_bootstrap.cc:654] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:38.475522 29704 tablet_bootstrap.cc:492] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: Bootstrap starting.
I20260808 17:05:38.476087 29702 log.cc:824] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:38.476122 29704 tablet_bootstrap.cc:654] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:38.476585 29704 log.cc:824] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:38.476869 29702 tablet_bootstrap.cc:492] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f: No bootstrap required, opened a new log
I20260808 17:05:38.476907 29707 tablet_bootstrap.cc:492] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: Bootstrap starting.
I20260808 17:05:38.476940 29702 ts_tablet_manager.cc:1397] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:05:38.477337 29704 tablet_bootstrap.cc:492] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: No bootstrap required, opened a new log
I20260808 17:05:38.477404 29704 ts_tablet_manager.cc:1397] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:05:38.478788 29702 raft_consensus.cc:348] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 } } peers { permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 } } peers { permanent_uuid: "c80f7e0a8fcb4474b62964f3b937530d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 38179 } }
I20260808 17:05:38.478963 29702 raft_consensus.cc:374] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:38.478997 29702 raft_consensus.cc:729] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408d222821964c57a083348d80240c8f, State: Initialized, Role: FOLLOWER
I20260808 17:05:38.479239 29702 consensus_queue.cc:260] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 } } peers { permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 } } peers { permanent_uuid: "c80f7e0a8fcb4474b62964f3b937530d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 38179 } }
I20260808 17:05:38.479198 29704 raft_consensus.cc:348] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 } } peers { permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 } } peers { permanent_uuid: "c80f7e0a8fcb4474b62964f3b937530d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 38179 } }
I20260808 17:05:38.479353 29704 raft_consensus.cc:374] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:38.479398 29704 raft_consensus.cc:729] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80f7e0a8fcb4474b62964f3b937530d, State: Initialized, Role: FOLLOWER
I20260808 17:05:38.479512 29702 ts_tablet_manager.cc:1428] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:05:38.479508 29704 consensus_queue.cc:260] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 } } peers { permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 } } peers { permanent_uuid: "c80f7e0a8fcb4474b62964f3b937530d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 38179 } }
I20260808 17:05:38.479633 29232 heartbeater.cc:503] Master 127.30.194.62:36813 was elected leader, sending a full tablet report...
I20260808 17:05:38.479755 29704 ts_tablet_manager.cc:1428] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:05:38.479765 29458 heartbeater.cc:503] Master 127.30.194.62:36813 was elected leader, sending a full tablet report...
I20260808 17:05:38.480424 29707 tablet_bootstrap.cc:654] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:38.480957 29707 log.cc:824] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:38.481562 29707 tablet_bootstrap.cc:492] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: No bootstrap required, opened a new log
I20260808 17:05:38.481614 29707 ts_tablet_manager.cc:1397] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260808 17:05:38.482957 29707 raft_consensus.cc:348] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 } } peers { permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 } } peers { permanent_uuid: "c80f7e0a8fcb4474b62964f3b937530d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 38179 } }
I20260808 17:05:38.483081 29707 raft_consensus.cc:374] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:38.483101 29707 raft_consensus.cc:729] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e6464f7aa04a07aae25748764f42c2, State: Initialized, Role: FOLLOWER
I20260808 17:05:38.483187 29707 consensus_queue.cc:260] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 } } peers { permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 } } peers { permanent_uuid: "c80f7e0a8fcb4474b62964f3b937530d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 38179 } }
I20260808 17:05:38.483423 29707 ts_tablet_manager.cc:1428] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:05:38.483481 29677 heartbeater.cc:503] Master 127.30.194.62:36813 was elected leader, sending a full tablet report...
I20260808 17:05:38.501748 29713 raft_consensus.cc:482] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:05:38.501843 29713 raft_consensus.cc:504] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 } } peers { permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 } } peers { permanent_uuid: "c80f7e0a8fcb4474b62964f3b937530d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 38179 } }
I20260808 17:05:38.502130 29713 leader_election.cc:288] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82e6464f7aa04a07aae25748764f42c2 (127.30.194.3:45781), 408d222821964c57a083348d80240c8f (127.30.194.1:37925)
I20260808 17:05:38.505704 29698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.502323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:34139 (local address 127.30.194.3:45781)
0808 17:05:38.502374 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.502376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.502384 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:38.502406 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.502407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.502415 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.502470 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:38.502651 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.503402 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.504800 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.505388 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.505458 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.505467 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.505607 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.505643 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:05:38.505718 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.502416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:35247 (local address 127.30.194.1:37925)
0808 17:05:38.502462 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.502464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.502549 (+ 85us) server_negotiation.cc:407] Connection header received
0808 17:05:38.502610 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.502611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.502619 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.502669 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:38.502757 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.503419 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.504801 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.505387 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.505459 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.505467 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.505607 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.505643 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:05:38.505728 29461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.502257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:45781 (local address 127.30.194.2:34139)
0808 17:05:38.502301 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:38.502312 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:38.502348 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:38.502522 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:38.502524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:38.502526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:38.502584 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.502585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.503480 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.503481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.504722 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.504725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.505455 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.505455 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.505538 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.505543 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:05:38.505597 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:05:38.505640 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":49}
I20260808 17:05:38.505734 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.502354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37925 (local address 127.30.194.2:35247)
0808 17:05:38.502500 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:38.502510 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:38.502559 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:38.502671 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:38.502672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:38.502673 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:38.502719 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.502720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.503477 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.503477 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.504729 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.504731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.505455 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.505455 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.505537 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.505543 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:05:38.505597 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:05:38.505640 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":84,"thread_start_us":81,"threads_started":1}
I20260808 17:05:38.506076 29154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11bc00273c914a52b14d8ba0dd96fcbd" candidate_uuid: "c80f7e0a8fcb4474b62964f3b937530d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408d222821964c57a083348d80240c8f" is_pre_election: true
I20260808 17:05:38.506098 29595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11bc00273c914a52b14d8ba0dd96fcbd" candidate_uuid: "c80f7e0a8fcb4474b62964f3b937530d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82e6464f7aa04a07aae25748764f42c2" is_pre_election: true
I20260808 17:05:38.506187 29154 raft_consensus.cc:2454] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c80f7e0a8fcb4474b62964f3b937530d in term 0.
I20260808 17:05:38.506211 29595 raft_consensus.cc:2454] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c80f7e0a8fcb4474b62964f3b937530d in term 0.
I20260808 17:05:38.506407 29278 leader_election.cc:302] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82e6464f7aa04a07aae25748764f42c2, c80f7e0a8fcb4474b62964f3b937530d; no voters:
I20260808 17:05:38.506527 29713 raft_consensus.cc:2781] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:05:38.506592 29713 raft_consensus.cc:482] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:05:38.506608 29713 raft_consensus.cc:3037] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:38.507221 29713 raft_consensus.cc:504] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 } } peers { permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 } } peers { permanent_uuid: "c80f7e0a8fcb4474b62964f3b937530d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 38179 } }
I20260808 17:05:38.507326 29713 leader_election.cc:288] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [CANDIDATE]: Term 1 election: Requested vote from peers 82e6464f7aa04a07aae25748764f42c2 (127.30.194.3:45781), 408d222821964c57a083348d80240c8f (127.30.194.1:37925)
I20260808 17:05:38.507454 29595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11bc00273c914a52b14d8ba0dd96fcbd" candidate_uuid: "c80f7e0a8fcb4474b62964f3b937530d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82e6464f7aa04a07aae25748764f42c2"
I20260808 17:05:38.507490 29154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11bc00273c914a52b14d8ba0dd96fcbd" candidate_uuid: "c80f7e0a8fcb4474b62964f3b937530d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408d222821964c57a083348d80240c8f"
I20260808 17:05:38.507525 29595 raft_consensus.cc:3037] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:38.507562 29154 raft_consensus.cc:3037] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:38.508165 29595 raft_consensus.cc:2454] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c80f7e0a8fcb4474b62964f3b937530d in term 1.
I20260808 17:05:38.508177 29154 raft_consensus.cc:2454] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c80f7e0a8fcb4474b62964f3b937530d in term 1.
I20260808 17:05:38.508308 29278 leader_election.cc:302] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82e6464f7aa04a07aae25748764f42c2, c80f7e0a8fcb4474b62964f3b937530d; no voters:
I20260808 17:05:38.508387 29713 raft_consensus.cc:2781] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:38.508522 29713 raft_consensus.cc:686] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 1 LEADER]: Becoming Leader. State: Replica: c80f7e0a8fcb4474b62964f3b937530d, State: Running, Role: LEADER
I20260808 17:05:38.508620 29713 consensus_queue.cc:237] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 } } peers { permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 } } peers { permanent_uuid: "c80f7e0a8fcb4474b62964f3b937530d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 38179 } }
I20260808 17:05:38.509327 28918 catalog_manager.cc:5495] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d reported cstate change: term changed from 0 to 1, leader changed from <none> to c80f7e0a8fcb4474b62964f3b937530d (127.30.194.2). New cstate: current_term: 1 leader_uuid: "c80f7e0a8fcb4474b62964f3b937530d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c80f7e0a8fcb4474b62964f3b937530d" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 38179 } health_report { overall_health: HEALTHY } } }
W20260808 17:05:38.512763 29459 tablet.cc:2369] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:05:38.515327 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.513405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47578 (local address 127.30.194.2:38179)
0808 17:05:38.513447 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.513448 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.513454 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:05:38.513477 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.513478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.513491 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.513539 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:05:38.513631 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.514046 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.514716 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.514891 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.514940 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.515044 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:05:38.515119 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:05:38.515180 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.515232 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.515242 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:05:38.516646 29154 raft_consensus.cc:1261] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f [term 1 FOLLOWER]: Refusing update from remote peer c80f7e0a8fcb4474b62964f3b937530d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:38.516678 29595 raft_consensus.cc:1261] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2 [term 1 FOLLOWER]: Refusing update from remote peer c80f7e0a8fcb4474b62964f3b937530d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:38.516902 29713 consensus_queue.cc:1035] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [LEADER]: Connected to new peer: Peer: permanent_uuid: "408d222821964c57a083348d80240c8f" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 37925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:38.517016 29721 consensus_queue.cc:1035] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [LEADER]: Connected to new peer: Peer: permanent_uuid: "82e6464f7aa04a07aae25748764f42c2" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:38.519163 29728 mvcc.cc:205] Tried to move back new op lower bound from 7316310992962326528 to 7316310992931819520. Current Snapshot: MvccSnapshot[applied={T|T < 7316310992962326528}]
I20260808 17:05:38.519207 29729 mvcc.cc:205] Tried to move back new op lower bound from 7316310992962326528 to 7316310992931819520. Current Snapshot: MvccSnapshot[applied={T|T < 7316310992962326528}]
I20260808 17:05:38.521304 28918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:38.521379 28918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:38.522053 28918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ebbdc9208b34b16998373930917c49b: Sending DeleteTablet for 3 replicas of tablet 11bc00273c914a52b14d8ba0dd96fcbd
I20260808 17:05:38.522372 29347 tablet_service.cc:1514] Processing DeleteTablet for tablet 11bc00273c914a52b14d8ba0dd96fcbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:38 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:47566
I20260808 17:05:38.522385 29559 tablet_service.cc:1514] Processing DeleteTablet for tablet 11bc00273c914a52b14d8ba0dd96fcbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:38 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:56574
I20260808 17:05:38.522569 29745 tablet_replica.cc:330] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: stopping tablet replica
I20260808 17:05:38.522595 29124 tablet_service.cc:1514] Processing DeleteTablet for tablet 11bc00273c914a52b14d8ba0dd96fcbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:38 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:49834
I20260808 17:05:38.522657 29745 raft_consensus.cc:2229] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:05:38.522773 29745 raft_consensus.cc:2258] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:38.522800 29750 tablet_replica.cc:330] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f: stopping tablet replica
I20260808 17:05:38.522897 29750 raft_consensus.cc:2229] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:38.523003 29750 raft_consensus.cc:2258] T 11bc00273c914a52b14d8ba0dd96fcbd P 408d222821964c57a083348d80240c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:38.523097 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 29020
I20260808 17:05:38.533963 29745 ts_tablet_manager.cc:1905] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:38.535887 29745 ts_tablet_manager.cc:1918] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:38.536059 29745 log.cc:1196] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/11bc00273c914a52b14d8ba0dd96fcbd
I20260808 17:05:38.536624 29749 tablet_replica.cc:330] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: stopping tablet replica
I20260808 17:05:38.536743 29745 ts_tablet_manager.cc:1939] T 11bc00273c914a52b14d8ba0dd96fcbd P c80f7e0a8fcb4474b62964f3b937530d: Deleting consensus metadata
I20260808 17:05:38.536764 29749 raft_consensus.cc:2229] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:38.536902 29749 raft_consensus.cc:2258] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:38.537217 28894 catalog_manager.cc:4841] TS c80f7e0a8fcb4474b62964f3b937530d (127.30.194.2:38179): tablet 11bc00273c914a52b14d8ba0dd96fcbd (table test-table [id=40be2f0060f44149bbdb0b6a762ea833]) successfully deleted
I20260808 17:05:38.538692 29749 ts_tablet_manager.cc:1905] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:38.540169 29749 ts_tablet_manager.cc:1918] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:38.540244 29749 log.cc:1196] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/11bc00273c914a52b14d8ba0dd96fcbd
I20260808 17:05:38.540493 29749 ts_tablet_manager.cc:1939] T 11bc00273c914a52b14d8ba0dd96fcbd P 82e6464f7aa04a07aae25748764f42c2: Deleting consensus metadata
I20260808 17:05:38.540921 28897 catalog_manager.cc:4841] TS 82e6464f7aa04a07aae25748764f42c2 (127.30.194.3:45781): tablet 11bc00273c914a52b14d8ba0dd96fcbd (table test-table [id=40be2f0060f44149bbdb0b6a762ea833]) successfully deleted
W20260808 17:05:38.543277 28897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:37925 (error 108)
I20260808 17:05:38.543776 29695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.543647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:37925 (local address 127.0.0.1:49850)
0808 17:05:38.543696 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:38.543742 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:37925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 17:05:38.543846 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 29240
W20260808 17:05:38.546357 28897 catalog_manager.cc:4584] TS 408d222821964c57a083348d80240c8f (127.30.194.1:37925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 11bc00273c914a52b14d8ba0dd96fcbd: Network error: Client connection negotiation failed: client connection to 127.30.194.1:37925: connect: Connection refused (error 111)
I20260808 17:05:38.552053 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 29464
I20260808 17:05:38.563115 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 28865
2026-08-08T17:05:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2028 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T17:05:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:38Z Disabled control of system clock
I20260808 17:05:38.594398 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:46601
--webserver_interface=127.30.194.62
--webserver_port=0
--builtin_ntp_servers=127.30.194.20:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:46601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:05:38.673322 29758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:38.673514 29758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:38.673533 29758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:38.674639 29758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:38.674677 29758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:38.674690 29758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:38.674712 29758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:38.674734 29758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:38.676014 29758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:46601
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:46601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:38.676245 29758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:38.676505 29758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:05:38.681052 29758 server_base.cc:1034] running on GCE node
W20260808 17:05:38.681046 29769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:38.681021 29771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:38.681007 29768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:38.681483 29758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:38.681753 29758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:38.682898 29758 hybrid_clock.cc:648] HybridClock initialized: now 1786208738682883 us; error 42 us; skew 500 ppm
I20260808 17:05:38.683992 29758 webserver.cc:460] Webserver started at http://127.30.194.62:44887/ using document root <none> and password file <none>
I20260808 17:05:38.684194 29758 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:38.684239 29758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:38.684361 29758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:38.685407 29758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "dec84a2880054fe58383856222bea223"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "0a37c8c60b7af36d2adb5f1153c43148"
server_key_iv: "8e1d5231b11836bd24b57137edb58025"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.685725 29758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "dec84a2880054fe58383856222bea223"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "0a37c8c60b7af36d2adb5f1153c43148"
server_key_iv: "8e1d5231b11836bd24b57137edb58025"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.686838 29758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:05:38.687461 29783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.687582 29758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.687657 29758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "dec84a2880054fe58383856222bea223"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "0a37c8c60b7af36d2adb5f1153c43148"
server_key_iv: "8e1d5231b11836bd24b57137edb58025"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.687726 29758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:38.719307 29758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:38.719563 29758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:38.719682 29758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:38.723551 29758 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:46601
I20260808 17:05:38.723587 29871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:46601 every 8 connection(s)
I20260808 17:05:38.723882 29758 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:38.724782 29872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:38.726933 29872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223: Bootstrap starting.
I20260808 17:05:38.727442 29872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:38.727662 29872 log.cc:824] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:38.728227 29872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223: No bootstrap required, opened a new log
I20260808 17:05:38.728503 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 29758
I20260808 17:05:38.728612 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:38.728799 31496 external_mini_cluster.cc:1428] Setting key 201de2ec2150d94700f1753b79ee1b62
I20260808 17:05:38.730360 29872 raft_consensus.cc:348] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec84a2880054fe58383856222bea223" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46601 } }
I20260808 17:05:38.730507 29872 raft_consensus.cc:374] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:38.730543 29872 raft_consensus.cc:729] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dec84a2880054fe58383856222bea223, State: Initialized, Role: FOLLOWER
I20260808 17:05:38.730664 29872 consensus_queue.cc:260] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec84a2880054fe58383856222bea223" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46601 } }
I20260808 17:05:38.730743 29872 raft_consensus.cc:388] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:38.730774 29872 raft_consensus.cc:482] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:38.730820 29872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:38.731290 29872 raft_consensus.cc:504] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec84a2880054fe58383856222bea223" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46601 } }
I20260808 17:05:38.731389 29872 leader_election.cc:302] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dec84a2880054fe58383856222bea223; no voters:
I20260808 17:05:38.731542 29872 leader_election.cc:288] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:38.731590 29881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:38.731732 29881 raft_consensus.cc:686] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [term 1 LEADER]: Becoming Leader. State: Replica: dec84a2880054fe58383856222bea223, State: Running, Role: LEADER
I20260808 17:05:38.731792 29872 sys_catalog.cc:564] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:38.731850 29881 consensus_queue.cc:237] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec84a2880054fe58383856222bea223" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46601 } }
I20260808 17:05:38.732146 29881 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dec84a2880054fe58383856222bea223. Latest consensus state: current_term: 1 leader_uuid: "dec84a2880054fe58383856222bea223" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec84a2880054fe58383856222bea223" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46601 } } }
I20260808 17:05:38.732378 29881 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:38.732311 29885 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dec84a2880054fe58383856222bea223" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec84a2880054fe58383856222bea223" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46601 } } }
I20260808 17:05:38.732801 29885 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:38.732975 29898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:38.733443 29898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:38.734997 29898 catalog_manager.cc:1347] Generated new cluster ID: 95f829e087f94000bd8aff7cf8144848
I20260808 17:05:38.735034 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.729303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.30.194.62:46601)
0808 17:05:38.729488 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.729495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.729518 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:05:38.729966 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.729973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.730026 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.730132 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:38.730398 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.731826 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.732923 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.733559 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.733675 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.734260 (+ 585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:38.734264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:38.734264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:38.734462 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:38.734503 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:38.734517 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:38.734571 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.734755 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.734829 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260808 17:05:38.735052 29898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:38.751331 29898 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:38.751780 29898 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:38.759207 29898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223: Generated new TSK 0
I20260808 17:05:38.759356 29898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:38.764470 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46601
--builtin_ntp_servers=127.30.194.20:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:05:38.844038 29918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:38.844228 29918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:38.844249 29918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:38.845360 29918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:38.845396 29918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:38.845419 29918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:05:38.846712 29918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:38.846916 29918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:38.847179 29918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:38.852833 29928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:38.852895 29926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:38.852952 29924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:38.853063 29918 server_base.cc:1034] running on GCE node
I20260808 17:05:38.853402 29918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:38.853621 29918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:38.854759 29918 hybrid_clock.cc:648] HybridClock initialized: now 1786208738854749 us; error 32 us; skew 500 ppm
I20260808 17:05:38.855902 29918 webserver.cc:460] Webserver started at http://127.30.194.1:39733/ using document root <none> and password file <none>
I20260808 17:05:38.856099 29918 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:38.856145 29918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:38.856293 29918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:38.857327 29918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "c5d5af82d83f423eb0c097eef7a64140"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "a35b3ef98f5bece5a5694245d733f98d"
server_key_iv: "7378b919e1e6779ec8eb8d4314413e8b"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.857642 29918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "c5d5af82d83f423eb0c097eef7a64140"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "a35b3ef98f5bece5a5694245d733f98d"
server_key_iv: "7378b919e1e6779ec8eb8d4314413e8b"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.858871 29918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 17:05:38.859596 29943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.859727 29918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:38.859808 29918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "c5d5af82d83f423eb0c097eef7a64140"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "a35b3ef98f5bece5a5694245d733f98d"
server_key_iv: "7378b919e1e6779ec8eb8d4314413e8b"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.859874 29918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:38.882826 29918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:38.883049 29918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:38.883141 29918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:38.883289 29918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:38.883528 29918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:38.883556 29918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.883579 29918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:38.883596 29918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.888937 29918 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:35403
I20260808 17:05:38.888974 30133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:35403 every 8 connection(s)
I20260808 17:05:38.889292 29918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:05:38.889931 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 29918
I20260808 17:05:38.890045 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:05:38.890259 31496 external_mini_cluster.cc:1428] Setting key 897114d3a571c6cf8f43686ffd19d3a7
I20260808 17:05:38.891196 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46601
--builtin_ntp_servers=127.30.194.20:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:05:38.894753 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.889933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:34519 (local address 127.30.194.62:46601)
0808 17:05:38.889995 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:38.889997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:38.890222 (+ 225us) server_negotiation.cc:407] Connection header received
0808 17:05:38.890760 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:38.890761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:38.890771 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:38.890823 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:38.891662 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.892143 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.893385 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.893692 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.893775 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.894511 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:38.894512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:38.894512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:38.894517 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:38.894534 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:38.894538 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:38.894585 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:38.894658 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 17:05:38.894694 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:05:38.894954 30139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:38.889914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:46601 (local address 127.30.194.1:34519)
0808 17:05:38.890103 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:38.890144 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:38.890678 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:38.890891 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:38.890895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:38.890905 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:38.891567 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.891573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.892222 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.892223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.893280 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:38.893283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:38.893771 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:38.893771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:38.893869 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:38.894360 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:05:38.894360 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:38.894396 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:05:38.894422 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:38.894594 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:38.894599 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:38.894602 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:38.894662 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:05:38.894757 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260808 17:05:38.895392 30134 heartbeater.cc:347] Connected to a master server at 127.30.194.62:46601
I20260808 17:05:38.895486 30134 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:38.895663 30134 heartbeater.cc:511] Master 127.30.194.62:46601 requested a full tablet report, sending...
I20260808 17:05:38.896181 29814 ts_manager.cc:194] Registered new tserver with Master: c5d5af82d83f423eb0c097eef7a64140 (127.30.194.1:35403)
I20260808 17:05:38.897046 29814 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.1:34519
W20260808 17:05:38.971486 30142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:38.971657 30142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:38.971676 30142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:38.972716 30142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:38.972746 30142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:38.972769 30142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:05:38.974076 30142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:38.974242 30142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:38.974462 30142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:38.978900 30158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:38.978914 30142 server_base.cc:1034] running on GCE node
W20260808 17:05:38.978888 30161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:38.978927 30157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:38.979285 30142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:38.979503 30142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:38.980626 30142 hybrid_clock.cc:648] HybridClock initialized: now 1786208738980618 us; error 27 us; skew 500 ppm
I20260808 17:05:38.981750 30142 webserver.cc:460] Webserver started at http://127.30.194.2:34609/ using document root <none> and password file <none>
I20260808 17:05:38.981961 30142 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:38.982004 30142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:38.982120 30142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:38.983067 30142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "0103859b1df94cb1a9848c09e0055055"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "b26cd9e200ff6a466c4ba0ef2236aeee"
server_key_iv: "a1b432f8a2d5d21bc2850c69ff4f0823"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.983378 30142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "0103859b1df94cb1a9848c09e0055055"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "b26cd9e200ff6a466c4ba0ef2236aeee"
server_key_iv: "a1b432f8a2d5d21bc2850c69ff4f0823"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.984520 30142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:05:38.985163 30169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.985306 30142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:38.985380 30142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "0103859b1df94cb1a9848c09e0055055"
format_stamp: "Formatted at 2026-08-08 17:05:38 on dist-test-slave-2j7r"
server_key: "b26cd9e200ff6a466c4ba0ef2236aeee"
server_key_iv: "a1b432f8a2d5d21bc2850c69ff4f0823"
server_key_version: "encryptionkey@0"
I20260808 17:05:38.985428 30142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:38.995886 30142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:38.996100 30142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:38.996189 30142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:38.996357 30142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:38.996609 30142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:38.996649 30142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:38.996685 30142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:38.996703 30142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:39.001847 30142 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:40847
I20260808 17:05:39.001912 30359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:40847 every 8 connection(s)
I20260808 17:05:39.002168 30142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:05:39.006793 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.002812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:48079 (local address 127.30.194.62:46601)
0808 17:05:39.002853 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:39.002855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:39.003080 (+ 225us) server_negotiation.cc:407] Connection header received
0808 17:05:39.003522 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:39.003523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:39.003533 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:39.003572 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:39.004162 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.004635 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.005609 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.005812 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.005875 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.006555 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:39.006556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:39.006557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:39.006561 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:39.006577 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:39.006581 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:39.006623 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:39.006680 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 17:05:39.006719 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:05:39.006929 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 30142
I20260808 17:05:39.006960 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.002775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:46601 (local address 127.30.194.2:48079)
0808 17:05:39.002971 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:39.003008 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:39.003436 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:39.003634 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:39.003637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:39.003645 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:39.004099 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.004103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.004695 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.004696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.005516 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.005519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.005882 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.005883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.005998 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.006427 (+ 429us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:05:39.006427 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:39.006454 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:05:39.006476 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:39.006628 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:39.006630 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:39.006633 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:39.006676 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 17:05:39.006755 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260808 17:05:39.007066 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:05:39.007233 31496 external_mini_cluster.cc:1428] Setting key 9846f3c82ad5406c46618ac5081c84c4
I20260808 17:05:39.007628 30361 heartbeater.cc:347] Connected to a master server at 127.30.194.62:46601
I20260808 17:05:39.007714 30361 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:39.007879 30361 heartbeater.cc:511] Master 127.30.194.62:46601 requested a full tablet report, sending...
I20260808 17:05:39.008083 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46601
--builtin_ntp_servers=127.30.194.20:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:05:39.009696 29814 ts_manager.cc:194] Registered new tserver with Master: 0103859b1df94cb1a9848c09e0055055 (127.30.194.2:40847)
I20260808 17:05:39.010053 29814 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.2:48079
W20260808 17:05:39.087357 30365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:39.087553 30365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:39.087582 30365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:39.088837 30365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:39.088876 30365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:39.088900 30365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:05:39.090291 30365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:39.090489 30365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:39.090760 30365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:39.095330 30374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:39.095392 30372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:39.095449 30365 server_base.cc:1034] running on GCE node
W20260808 17:05:39.095397 30377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:39.095759 30365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:39.095988 30365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:39.097141 30365 hybrid_clock.cc:648] HybridClock initialized: now 1786208739097135 us; error 33 us; skew 500 ppm
I20260808 17:05:39.098201 30365 webserver.cc:460] Webserver started at http://127.30.194.3:46867/ using document root <none> and password file <none>
I20260808 17:05:39.098413 30365 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:39.098459 30365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:39.098584 30365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:39.099519 30365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "1a0fa9797a41417d83ba8a722ecd424b"
format_stamp: "Formatted at 2026-08-08 17:05:39 on dist-test-slave-2j7r"
server_key: "f5da661b46297d84c4e35a92af1ed43f"
server_key_iv: "f4fa59abf03a6fdb2f2bdc24faab0104"
server_key_version: "encryptionkey@0"
I20260808 17:05:39.099835 30365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "1a0fa9797a41417d83ba8a722ecd424b"
format_stamp: "Formatted at 2026-08-08 17:05:39 on dist-test-slave-2j7r"
server_key: "f5da661b46297d84c4e35a92af1ed43f"
server_key_iv: "f4fa59abf03a6fdb2f2bdc24faab0104"
server_key_version: "encryptionkey@0"
I20260808 17:05:39.100980 30365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:05:39.101672 30390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:39.101810 30365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:39.101891 30365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "1a0fa9797a41417d83ba8a722ecd424b"
format_stamp: "Formatted at 2026-08-08 17:05:39 on dist-test-slave-2j7r"
server_key: "f5da661b46297d84c4e35a92af1ed43f"
server_key_iv: "f4fa59abf03a6fdb2f2bdc24faab0104"
server_key_version: "encryptionkey@0"
I20260808 17:05:39.101955 30365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:39.111593 30365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:39.111820 30365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:39.111936 30365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:39.112118 30365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:39.112402 30365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:39.112434 30365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:39.112460 30365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:39.112486 30365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:39.118002 30365 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:37537
I20260808 17:05:39.118060 30578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:37537 every 8 connection(s)
I20260808 17:05:39.118316 30365 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:05:39.122651 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.118926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:45723 (local address 127.30.194.62:46601)
0808 17:05:39.119002 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:39.119004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:39.119170 (+ 166us) server_negotiation.cc:407] Connection header received
0808 17:05:39.119574 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:39.119575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:39.119585 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:39.119635 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:39.120194 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.120675 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.121588 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.121772 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.121823 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.122449 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:39.122450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:39.122450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:39.122455 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:39.122471 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:39.122475 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:39.122522 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:39.122571 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 17:05:39.122592 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 17:05:39.122815 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.118902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:46601 (local address 127.30.194.3:45723)
0808 17:05:39.119066 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:39.119101 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:39.119504 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:39.119680 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:39.119683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:39.119690 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:39.120128 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.120131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.120734 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.120734 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.121515 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.121517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.121831 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.121832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.121925 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.122324 (+ 399us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:05:39.122330 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:39.122357 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:05:39.122377 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:39.122524 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:39.122526 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:39.122528 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:05:39.122572 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 17:05:39.122639 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260808 17:05:39.123230 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 30365
I20260808 17:05:39.123306 30579 heartbeater.cc:347] Connected to a master server at 127.30.194.62:46601
I20260808 17:05:39.123353 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:05:39.123407 30579 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:39.123553 31496 external_mini_cluster.cc:1428] Setting key dff04c316c0357aeeec970b88534fe15
I20260808 17:05:39.123564 30579 heartbeater.cc:511] Master 127.30.194.62:46601 requested a full tablet report, sending...
I20260808 17:05:39.123940 29813 ts_manager.cc:194] Registered new tserver with Master: 1a0fa9797a41417d83ba8a722ecd424b (127.30.194.3:37537)
I20260808 17:05:39.124352 29813 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.3:45723
I20260808 17:05:39.124378 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:05:39.127907 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.125650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.30.194.62:46601)
0808 17:05:39.125686 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:39.125687 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:39.125762 (+ 75us) server_negotiation.cc:407] Connection header received
0808 17:05:39.125804 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:39.125804 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:39.125812 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:39.125852 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:39.125981 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.126426 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.127069 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.127249 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.127290 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.127715 (+ 425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:39.127716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:39.127717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:39.127721 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:39.127732 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:39.127735 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:39.127794 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:39.127832 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 17:05:39.127841 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:05:39.129261 29813 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:05:39.130683 29813 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:05:39.136700 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.132746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59056 (local address 127.30.194.3:37537)
0808 17:05:39.133115 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:39.133119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:39.133127 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:39.133158 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:39.133160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:39.133174 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:39.133278 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:39.133413 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.134489 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.135680 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.136392 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.136462 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.136542 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:39.136620 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 17:05:39.136649 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":67,"threads_started":1}
I20260808 17:05:39.136731 30589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.132531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:37537 (local address 127.0.0.1:59056)
0808 17:05:39.132791 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:39.132815 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:39.132876 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:39.133283 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:39.133284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:39.133286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:39.133330 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.133332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.134573 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.134574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.135604 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.135606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.136458 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.136459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.136534 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.136547 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:05:39.136612 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 17:05:39.136660 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":105,"threads_started":1}
I20260808 17:05:39.137131 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.132527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:40847 (local address 127.0.0.1:59950)
0808 17:05:39.132779 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:39.132814 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:39.132876 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:39.133347 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:39.133348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:39.133349 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:39.133381 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.133382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.134361 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.134362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.135588 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.135591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.136905 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.136906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.136963 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.136967 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:05:39.137012 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:05:39.137049 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":45,"threads_started":1}
I20260808 17:05:39.137192 30597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.132764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59950 (local address 127.30.194.2:40847)
0808 17:05:39.133034 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:39.133038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:39.133046 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:39.133078 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:39.133079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:39.133114 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:39.133187 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:39.133457 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.134279 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.135671 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.136396 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.136462 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.136543 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:39.137048 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 17:05:39.137108 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":87,"threads_started":1}
I20260808 17:05:39.137413 30478 tablet_service.cc:1467] Processing CreateTablet for tablet d614c231acce4c2cb8c705d31ece9e99 (DEFAULT_TABLE table=test-table [id=6d2f11af954b4e69bc7ab8fb4aec82b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:39.137732 30478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d614c231acce4c2cb8c705d31ece9e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:39.140092 30598 tablet_bootstrap.cc:492] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: Bootstrap starting.
I20260808 17:05:39.140751 30598 tablet_bootstrap.cc:654] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:39.140793 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.132531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:35403 (local address 127.0.0.1:47038)
0808 17:05:39.132945 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:39.132955 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:39.133007 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:39.133210 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:39.133213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:39.133214 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:39.133274 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.133276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.135680 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.135681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.136331 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.136333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.140571 (+ 4238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.140572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.140630 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.140635 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:05:39.140690 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:05:39.140727 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":6,"thread_start_us":128,"threads_started":1}
I20260808 17:05:39.140820 30595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.132627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47038 (local address 127.30.194.1:35403)
0808 17:05:39.133012 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:39.133017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:39.133028 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:05:39.133066 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:39.133068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:39.133106 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:39.133187 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:39.133359 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.134490 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.136410 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.140494 (+ 4084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.140556 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.140641 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:39.140709 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 17:05:39.140745 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":81,"threads_started":1}
I20260808 17:05:39.140723 30246 tablet_service.cc:1467] Processing CreateTablet for tablet d614c231acce4c2cb8c705d31ece9e99 (DEFAULT_TABLE table=test-table [id=6d2f11af954b4e69bc7ab8fb4aec82b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:39.141065 30246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d614c231acce4c2cb8c705d31ece9e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:39.141592 30021 tablet_service.cc:1467] Processing CreateTablet for tablet d614c231acce4c2cb8c705d31ece9e99 (DEFAULT_TABLE table=test-table [id=6d2f11af954b4e69bc7ab8fb4aec82b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:39.141897 30021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d614c231acce4c2cb8c705d31ece9e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:39.144213 30601 tablet_bootstrap.cc:492] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055: Bootstrap starting.
I20260808 17:05:39.144881 30601 tablet_bootstrap.cc:654] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:39.144876 30603 tablet_bootstrap.cc:492] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140: Bootstrap starting.
I20260808 17:05:39.145049 30598 log.cc:824] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:39.145251 30601 log.cc:824] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:39.145499 30603 tablet_bootstrap.cc:654] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:39.145817 30598 tablet_bootstrap.cc:492] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: No bootstrap required, opened a new log
I20260808 17:05:39.145794 30603 log.cc:824] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:39.145876 30598 ts_tablet_manager.cc:1397] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260808 17:05:39.145987 30601 tablet_bootstrap.cc:492] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055: No bootstrap required, opened a new log
I20260808 17:05:39.146044 30601 ts_tablet_manager.cc:1397] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:05:39.146566 30603 tablet_bootstrap.cc:492] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140: No bootstrap required, opened a new log
I20260808 17:05:39.146621 30603 ts_tablet_manager.cc:1397] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:05:39.147666 30598 raft_consensus.cc:348] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 } } peers { permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 } } peers { permanent_uuid: "0103859b1df94cb1a9848c09e0055055" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40847 } }
I20260808 17:05:39.147789 30598 raft_consensus.cc:374] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:39.147807 30598 raft_consensus.cc:729] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0fa9797a41417d83ba8a722ecd424b, State: Initialized, Role: FOLLOWER
I20260808 17:05:39.147895 30598 consensus_queue.cc:260] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 } } peers { permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 } } peers { permanent_uuid: "0103859b1df94cb1a9848c09e0055055" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40847 } }
I20260808 17:05:39.148119 30598 ts_tablet_manager.cc:1428] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:05:39.148200 30579 heartbeater.cc:503] Master 127.30.194.62:46601 was elected leader, sending a full tablet report...
I20260808 17:05:39.148351 30601 raft_consensus.cc:348] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 } } peers { permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 } } peers { permanent_uuid: "0103859b1df94cb1a9848c09e0055055" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40847 } }
I20260808 17:05:39.148509 30601 raft_consensus.cc:374] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:39.148551 30601 raft_consensus.cc:729] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0103859b1df94cb1a9848c09e0055055, State: Initialized, Role: FOLLOWER
I20260808 17:05:39.148658 30601 consensus_queue.cc:260] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 } } peers { permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 } } peers { permanent_uuid: "0103859b1df94cb1a9848c09e0055055" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40847 } }
I20260808 17:05:39.148898 30601 ts_tablet_manager.cc:1428] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:05:39.149055 30361 heartbeater.cc:503] Master 127.30.194.62:46601 was elected leader, sending a full tablet report...
I20260808 17:05:39.149240 30603 raft_consensus.cc:348] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 } } peers { permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 } } peers { permanent_uuid: "0103859b1df94cb1a9848c09e0055055" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40847 } }
I20260808 17:05:39.149382 30603 raft_consensus.cc:374] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:39.149426 30603 raft_consensus.cc:729] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d5af82d83f423eb0c097eef7a64140, State: Initialized, Role: FOLLOWER
I20260808 17:05:39.149529 30603 consensus_queue.cc:260] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 } } peers { permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 } } peers { permanent_uuid: "0103859b1df94cb1a9848c09e0055055" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40847 } }
I20260808 17:05:39.149765 30603 ts_tablet_manager.cc:1428] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:05:39.149878 30134 heartbeater.cc:503] Master 127.30.194.62:46601 was elected leader, sending a full tablet report...
I20260808 17:05:39.244208 30610 raft_consensus.cc:482] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:05:39.244359 30610 raft_consensus.cc:504] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 } } peers { permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 } } peers { permanent_uuid: "0103859b1df94cb1a9848c09e0055055" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40847 } }
I20260808 17:05:39.244691 30610 leader_election.cc:288] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a0fa9797a41417d83ba8a722ecd424b (127.30.194.3:37537), c5d5af82d83f423eb0c097eef7a64140 (127.30.194.1:35403)
I20260808 17:05:39.248447 30595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.244966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:52755 (local address 127.30.194.1:35403)
0808 17:05:39.245027 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:39.245028 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:39.245037 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:05:39.245067 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:39.245068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:39.245077 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:39.245169 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:39.245337 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.246103 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.247500 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.248042 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.248120 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.248131 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:39.248336 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 17:05:39.248374 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:05:39.248448 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.244974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:38379 (local address 127.30.194.3:37537)
0808 17:05:39.245018 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:39.245021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:39.245140 (+ 119us) server_negotiation.cc:407] Connection header received
0808 17:05:39.245202 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:39.245203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:39.245211 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:39.245258 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:39.245391 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.246101 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.247496 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.248063 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.248125 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.248131 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:39.248334 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 17:05:39.248374 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:05:39.248454 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.244895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:35403 (local address 127.30.194.2:52755)
0808 17:05:39.244957 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:39.244969 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:39.245014 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:39.245180 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:39.245181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:39.245183 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:39.245257 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.245260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.246180 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.246181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.247410 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.247413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.248156 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.248158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.248246 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.248252 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:05:39.248317 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 17:05:39.248357 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":5}
I20260808 17:05:39.248466 30613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.244895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:37537 (local address 127.30.194.2:38379)
0808 17:05:39.245072 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:39.245080 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:39.245148 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:39.245268 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:39.245268 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:39.245270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:39.245331 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.245332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.246180 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.246181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.247416 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:39.247417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.248159 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.248159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:39.248246 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.248253 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 17:05:39.248312 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:05:39.248354 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":48,"thread_start_us":104,"threads_started":1}
I20260808 17:05:39.248808 30506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d614c231acce4c2cb8c705d31ece9e99" candidate_uuid: "0103859b1df94cb1a9848c09e0055055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0fa9797a41417d83ba8a722ecd424b" is_pre_election: true
I20260808 17:05:39.248983 30506 raft_consensus.cc:2454] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0103859b1df94cb1a9848c09e0055055 in term 0.
I20260808 17:05:39.249179 30061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d614c231acce4c2cb8c705d31ece9e99" candidate_uuid: "0103859b1df94cb1a9848c09e0055055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5d5af82d83f423eb0c097eef7a64140" is_pre_election: true
I20260808 17:05:39.249255 30173 leader_election.cc:302] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0103859b1df94cb1a9848c09e0055055, 1a0fa9797a41417d83ba8a722ecd424b; no voters:
I20260808 17:05:39.249323 30061 raft_consensus.cc:2454] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0103859b1df94cb1a9848c09e0055055 in term 0.
I20260808 17:05:39.249377 30610 raft_consensus.cc:2781] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:05:39.249432 30610 raft_consensus.cc:482] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:05:39.249452 30610 raft_consensus.cc:3037] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:39.250138 30610 raft_consensus.cc:504] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 } } peers { permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 } } peers { permanent_uuid: "0103859b1df94cb1a9848c09e0055055" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40847 } }
I20260808 17:05:39.250260 30610 leader_election.cc:288] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [CANDIDATE]: Term 1 election: Requested vote from peers 1a0fa9797a41417d83ba8a722ecd424b (127.30.194.3:37537), c5d5af82d83f423eb0c097eef7a64140 (127.30.194.1:35403)
I20260808 17:05:39.250437 30061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d614c231acce4c2cb8c705d31ece9e99" candidate_uuid: "0103859b1df94cb1a9848c09e0055055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5d5af82d83f423eb0c097eef7a64140"
I20260808 17:05:39.250486 30506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d614c231acce4c2cb8c705d31ece9e99" candidate_uuid: "0103859b1df94cb1a9848c09e0055055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0fa9797a41417d83ba8a722ecd424b"
I20260808 17:05:39.250505 30061 raft_consensus.cc:3037] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:39.250571 30506 raft_consensus.cc:3037] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:39.251186 30061 raft_consensus.cc:2454] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0103859b1df94cb1a9848c09e0055055 in term 1.
I20260808 17:05:39.251269 30506 raft_consensus.cc:2454] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0103859b1df94cb1a9848c09e0055055 in term 1.
I20260808 17:05:39.251358 30170 leader_election.cc:302] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0103859b1df94cb1a9848c09e0055055, c5d5af82d83f423eb0c097eef7a64140; no voters:
I20260808 17:05:39.251444 30610 raft_consensus.cc:2781] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:39.251570 30610 raft_consensus.cc:686] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 1 LEADER]: Becoming Leader. State: Replica: 0103859b1df94cb1a9848c09e0055055, State: Running, Role: LEADER
I20260808 17:05:39.251648 30610 consensus_queue.cc:237] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 } } peers { permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 } } peers { permanent_uuid: "0103859b1df94cb1a9848c09e0055055" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40847 } }
I20260808 17:05:39.252457 29813 catalog_manager.cc:5495] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0103859b1df94cb1a9848c09e0055055 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "0103859b1df94cb1a9848c09e0055055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0103859b1df94cb1a9848c09e0055055" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 40847 } health_report { overall_health: HEALTHY } } }
W20260808 17:05:39.252610 30362 tablet.cc:2369] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:05:39.280109 30597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.277980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.30.194.2:40847)
0808 17:05:39.278035 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:39.278037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:39.278046 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:05:39.278105 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:39.278106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:39.278124 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:39.278182 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:05:39.278307 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.278756 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.279429 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:39.279601 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:39.279641 (+ 40us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:39.279758 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:05:39.279851 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:05:39.279918 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:39.280022 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 17:05:39.280030 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 17:05:39.281802 30061 raft_consensus.cc:1261] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140 [term 1 FOLLOWER]: Refusing update from remote peer 0103859b1df94cb1a9848c09e0055055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:39.281808 30506 raft_consensus.cc:1261] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b [term 1 FOLLOWER]: Refusing update from remote peer 0103859b1df94cb1a9848c09e0055055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:39.282161 30610 consensus_queue.cc:1035] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a0fa9797a41417d83ba8a722ecd424b" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 37537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:39.282315 30614 consensus_queue.cc:1035] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5d5af82d83f423eb0c097eef7a64140" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:39.286893 29811 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:39.286978 29811 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:39.287681 29811 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dec84a2880054fe58383856222bea223: Sending DeleteTablet for 3 replicas of tablet d614c231acce4c2cb8c705d31ece9e99
I20260808 17:05:39.288003 30246 tablet_service.cc:1514] Processing DeleteTablet for tablet d614c231acce4c2cb8c705d31ece9e99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:39 UTC) from {username='slave'} at 127.0.0.1:59950
I20260808 17:05:39.288021 30478 tablet_service.cc:1514] Processing DeleteTablet for tablet d614c231acce4c2cb8c705d31ece9e99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:39 UTC) from {username='slave'} at 127.0.0.1:59056
I20260808 17:05:39.288208 30021 tablet_service.cc:1514] Processing DeleteTablet for tablet d614c231acce4c2cb8c705d31ece9e99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:39 UTC) from {username='slave'} at 127.0.0.1:47038
I20260808 17:05:39.288251 30632 tablet_replica.cc:330] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: stopping tablet replica
I20260808 17:05:39.288321 30632 raft_consensus.cc:2229] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:39.288393 30632 raft_consensus.cc:2258] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:39.288458 30633 tablet_replica.cc:330] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140: stopping tablet replica
I20260808 17:05:39.288538 30633 raft_consensus.cc:2229] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:39.288622 30633 raft_consensus.cc:2258] T d614c231acce4c2cb8c705d31ece9e99 P c5d5af82d83f423eb0c097eef7a64140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:39.288731 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 29918
I20260808 17:05:39.289355 30632 ts_tablet_manager.cc:1905] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:39.290649 30632 ts_tablet_manager.cc:1918] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:39.290699 30632 log.cc:1196] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/d614c231acce4c2cb8c705d31ece9e99
I20260808 17:05:39.290864 30632 ts_tablet_manager.cc:1939] T d614c231acce4c2cb8c705d31ece9e99 P 1a0fa9797a41417d83ba8a722ecd424b: Deleting consensus metadata
I20260808 17:05:39.291177 29789 catalog_manager.cc:4841] TS 1a0fa9797a41417d83ba8a722ecd424b (127.30.194.3:37537): tablet d614c231acce4c2cb8c705d31ece9e99 (table test-table [id=6d2f11af954b4e69bc7ab8fb4aec82b2]) successfully deleted
W20260808 17:05:39.300662 29785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:35403 (error 108)
I20260808 17:05:39.300910 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 30142
I20260808 17:05:39.300993 30630 tablet_replica.cc:330] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055: stopping tablet replica
I20260808 17:05:39.301080 30630 raft_consensus.cc:2229] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:05:39.301191 30630 raft_consensus.cc:2258] T d614c231acce4c2cb8c705d31ece9e99 P 0103859b1df94cb1a9848c09e0055055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:39.304612 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.303143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:35403 (local address 127.0.0.1:47048)
0808 17:05:39.304521 (+ 1378us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:39.304576 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:35403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1351}
W20260808 17:05:39.304776 29785 catalog_manager.cc:4584] TS c5d5af82d83f423eb0c097eef7a64140 (127.30.194.1:35403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d614c231acce4c2cb8c705d31ece9e99: Network error: Client connection negotiation failed: client connection to 127.30.194.1:35403: connect: Connection refused (error 111)
I20260808 17:05:39.310386 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:39.310285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:40847 (local address 127.0.0.1:59972)
0808 17:05:39.310337 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:39.310360 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.2:40847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 17:05:39.310412 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 30365
I20260808 17:05:39.323043 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 29758
2026-08-08T17:05:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (757 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:39 dist-test-slave-2j7r krb5kdc[30638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:39 dist-test-slave-2j7r krb5kdc[30638](info): set up 2 sockets
Aug 08 17:05:39 dist-test-slave-2j7r krb5kdc[30638](info): commencing operation
krb5kdc: starting...
W20260808 17:05:40.458657 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:40 dist-test-slave-2j7r krb5kdc[30638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:05:40.565469 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:46049
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:46049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:40.643764 30671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:40.643965 30671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:40.643994 30671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:40.645141 30671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:40.645180 30671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:40.645200 30671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:40.645216 30671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:40.645239 30671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:40.646698 30671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:46049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:46049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:40.646931 30671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:40.647112 30671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:05:40.651615 30671 server_base.cc:1034] running on GCE node
W20260808 17:05:40.651567 30681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:40.651587 30686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:40.651583 30679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:40.651913 30671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:40.652163 30671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:40.653303 30671 hybrid_clock.cc:648] HybridClock initialized: now 1786208740653289 us; error 29 us; skew 500 ppm
Aug 08 17:05:40 dist-test-slave-2j7r krb5kdc[30638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:40.655522 30671 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:05:40.655893 30671 webserver.cc:460] Webserver started at http://127.30.194.62:44661/ using document root <none> and password file <none>
I20260808 17:05:40.656086 30671 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:40.656131 30671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:40.656251 30671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:40.657226 30671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "0d73dac3c2c446c6891da44cc64f3a0f"
format_stamp: "Formatted at 2026-08-08 17:05:40 on dist-test-slave-2j7r"
server_key: "ec604ceb90f72dbf010110b4cd256c9e"
server_key_iv: "6f83842608ac1571e0e7c257e141e54d"
server_key_version: "encryptionkey@0"
I20260808 17:05:40.657541 30671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "0d73dac3c2c446c6891da44cc64f3a0f"
format_stamp: "Formatted at 2026-08-08 17:05:40 on dist-test-slave-2j7r"
server_key: "ec604ceb90f72dbf010110b4cd256c9e"
server_key_iv: "6f83842608ac1571e0e7c257e141e54d"
server_key_version: "encryptionkey@0"
I20260808 17:05:40.658687 30671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:05:40.659318 30695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:40.659466 30671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:40.659546 30671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "0d73dac3c2c446c6891da44cc64f3a0f"
format_stamp: "Formatted at 2026-08-08 17:05:40 on dist-test-slave-2j7r"
server_key: "ec604ceb90f72dbf010110b4cd256c9e"
server_key_iv: "6f83842608ac1571e0e7c257e141e54d"
server_key_version: "encryptionkey@0"
I20260808 17:05:40.659602 30671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:40.682354 30671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:40.683925 30671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:40.684055 30671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:40.687649 30671 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:46049
I20260808 17:05:40.687686 30779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:46049 every 8 connection(s)
I20260808 17:05:40.687989 30671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:40.688764 30780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:40.689472 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 30671
I20260808 17:05:40.689570 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:40.689740 31496 external_mini_cluster.cc:1428] Setting key c64a66c1badd07952b2b3a9ee70f46b4
I20260808 17:05:40.691614 30780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f: Bootstrap starting.
I20260808 17:05:40.692169 30780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:40.692432 30780 log.cc:824] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:40.693135 30780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f: No bootstrap required, opened a new log
I20260808 17:05:40.695516 30780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d73dac3c2c446c6891da44cc64f3a0f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46049 } }
I20260808 17:05:40.695644 30780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:40.695667 30780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d73dac3c2c446c6891da44cc64f3a0f, State: Initialized, Role: FOLLOWER
I20260808 17:05:40.695778 30780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d73dac3c2c446c6891da44cc64f3a0f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46049 } }
I20260808 17:05:40.695844 30780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:40.695866 30780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:40.695911 30780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [term 0 FOLLOWER]: Advancing to term 1
Aug 08 17:05:40 dist-test-slave-2j7r krb5kdc[30638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:40.696429 30780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d73dac3c2c446c6891da44cc64f3a0f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46049 } }
I20260808 17:05:40.696516 30780 leader_election.cc:302] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d73dac3c2c446c6891da44cc64f3a0f; no voters:
I20260808 17:05:40.696671 30780 leader_election.cc:288] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:40.696713 30788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:40.696868 30788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [term 1 LEADER]: Becoming Leader. State: Replica: 0d73dac3c2c446c6891da44cc64f3a0f, State: Running, Role: LEADER
I20260808 17:05:40.696931 30780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:40.696980 30788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d73dac3c2c446c6891da44cc64f3a0f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46049 } }
I20260808 17:05:40.697376 30793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d73dac3c2c446c6891da44cc64f3a0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d73dac3c2c446c6891da44cc64f3a0f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46049 } } }
I20260808 17:05:40.697464 30793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:40.697674 30802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:40.697387 30794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d73dac3c2c446c6891da44cc64f3a0f. Latest consensus state: current_term: 1 leader_uuid: "0d73dac3c2c446c6891da44cc64f3a0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d73dac3c2c446c6891da44cc64f3a0f" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 46049 } } }
I20260808 17:05:40.697793 30794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:40.698236 30802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:40.699889 30802 catalog_manager.cc:1347] Generated new cluster ID: 295da59581a648c99ab37e8daa870a90
I20260808 17:05:40.699944 30802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:40.700557 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:40.690203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57742 (local address 127.30.194.62:46049)
0808 17:05:40.690359 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:40.690366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:40.690389 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:05:40.690764 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:40.690770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:40.690812 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:40.690922 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:40.691635 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:40.692529 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:40.693478 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:40.693875 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:40.693953 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:40.696433 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:40.696438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:40.696438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:40.696449 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:40.698893 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:40.699276 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:40.699280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:40.699280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:40.699381 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:40.699695 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:40.699696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:40.699696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:40.699972 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:40.700096 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:40.700223 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 17:05:40.700362 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260808 17:05:40.732669 30802 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:40.733387 30802 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:40.736524 30802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f: Generated new TSK 0
I20260808 17:05:40.736725 30802 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
Entry for principal kudu/127.30.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab.
I20260808 17:05:40.765689 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46049
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:40.865160 30820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:40.865417 30820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:40.865450 30820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:40.866796 30820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:40.866851 30820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:40.866888 30820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:05:40.868662 30820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.1.keytab
--principal=kudu/127.30.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:40.868881 30820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:40.869138 30820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:40.873809 30833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:40.873821 30829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:40.873871 30828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:40.874094 30820 server_base.cc:1034] running on GCE node
I20260808 17:05:40.874276 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:40.874473 30820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:40.875600 30820 hybrid_clock.cc:648] HybridClock initialized: now 1786208740875592 us; error 26 us; skew 500 ppm
Aug 08 17:05:40 dist-test-slave-2j7r krb5kdc[30638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:40.877787 30820 init.cc:377] Logged in from keytab as kudu/127.30.194.1@KRBTEST.COM (short username kudu)
I20260808 17:05:40.878188 30820 webserver.cc:460] Webserver started at http://127.30.194.1:46541/ using document root <none> and password file <none>
I20260808 17:05:40.878386 30820 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:40.878433 30820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:40.878553 30820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:40.879470 30820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "5745fa9b7ca84055a1f847ca1870b96e"
format_stamp: "Formatted at 2026-08-08 17:05:40 on dist-test-slave-2j7r"
server_key: "40a29061fa4a67c206e80c7f829b52db"
server_key_iv: "f259d94ba0d3a83c046325bb4eb269f2"
server_key_version: "encryptionkey@0"
I20260808 17:05:40.879777 30820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "5745fa9b7ca84055a1f847ca1870b96e"
format_stamp: "Formatted at 2026-08-08 17:05:40 on dist-test-slave-2j7r"
server_key: "40a29061fa4a67c206e80c7f829b52db"
server_key_iv: "f259d94ba0d3a83c046325bb4eb269f2"
server_key_version: "encryptionkey@0"
I20260808 17:05:40.880939 30820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:05:40.881608 30843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:40.881770 30820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:40.881851 30820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "5745fa9b7ca84055a1f847ca1870b96e"
format_stamp: "Formatted at 2026-08-08 17:05:40 on dist-test-slave-2j7r"
server_key: "40a29061fa4a67c206e80c7f829b52db"
server_key_iv: "f259d94ba0d3a83c046325bb4eb269f2"
server_key_version: "encryptionkey@0"
I20260808 17:05:40.881913 30820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:40.908893 30820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:40.910442 30820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:40.910567 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:40.910778 30820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:40.911046 30820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:40.911078 30820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:40.911105 30820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:40.911123 30820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:40.916363 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:43561
I20260808 17:05:40.916397 31041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:43561 every 8 connection(s)
I20260808 17:05:40.916663 30820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:05:40.920421 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 30820
I20260808 17:05:40.920533 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:05:40.920748 31496 external_mini_cluster.cc:1428] Setting key 6a88ba4bd0604de82cc22655a8b178f1
Aug 08 17:05:40 dist-test-slave-2j7r krb5kdc[30638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.1@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:40.925210 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:40.917317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:48371 (local address 127.30.194.62:46049)
0808 17:05:40.917371 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:40.917373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:40.917544 (+ 171us) server_negotiation.cc:407] Connection header received
0808 17:05:40.917988 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:40.917989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:40.917999 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:40.918046 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:40.918849 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:40.919334 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:40.920189 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:40.920383 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:40.920426 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:40.922602 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:40.922603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:40.922604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:40.922610 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:40.924014 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:40.924375 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:40.924376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:40.924376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:40.924437 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:40.924625 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:40.924626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:40.924626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:40.924749 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:40.924808 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:40.925016 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 17:05:40.925123 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:05:40.925261 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:40.917300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:46049 (local address 127.30.194.1:48371)
0808 17:05:40.917437 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:40.917470 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:40.917910 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:40.918095 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:40.918098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:40.918615 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:40.918784 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:40.918788 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:40.919393 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:40.919394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:40.920131 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:40.920133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:40.920452 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:40.920453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:40.920592 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:40.921035 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:40.921043 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:40.922518 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:40.924086 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:40.924092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:40.924094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:40.924296 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:40.924489 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:40.924490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:40.924490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:40.924575 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:40.924806 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:40.924809 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:40.924907 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 17:05:40.924977 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 17:05:40.925104 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260808 17:05:40.925717 31043 heartbeater.cc:347] Connected to a master server at 127.30.194.62:46049
I20260808 17:05:40.925815 31043 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:40.926013 31043 heartbeater.cc:511] Master 127.30.194.62:46049 requested a full tablet report, sending...
I20260808 17:05:40.926551 30728 ts_manager.cc:194] Registered new tserver with Master: 5745fa9b7ca84055a1f847ca1870b96e (127.30.194.1:43561)
I20260808 17:05:40.927135 30728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.1@KRBTEST.COM'} at 127.30.194.1:48371
WARNING: no policy specified for kudu/127.30.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
Entry for principal kudu/127.30.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab.
I20260808 17:05:40.949642 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46049
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:41.036383 31048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:41.036556 31048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:41.036576 31048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:41.037747 31048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:41.037798 31048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:41.037825 31048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:05:41.039206 31048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.2.keytab
--principal=kudu/127.30.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:41.039409 31048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:41.039608 31048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:41.044093 31063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.044085 31060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.044082 31061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:41.044304 31048 server_base.cc:1034] running on GCE node
I20260808 17:05:41.044509 31048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:41.044718 31048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:41.045857 31048 hybrid_clock.cc:648] HybridClock initialized: now 1786208741045845 us; error 31 us; skew 500 ppm
Aug 08 17:05:41 dist-test-slave-2j7r krb5kdc[30638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:41.048121 31048 init.cc:377] Logged in from keytab as kudu/127.30.194.2@KRBTEST.COM (short username kudu)
I20260808 17:05:41.048489 31048 webserver.cc:460] Webserver started at http://127.30.194.2:36381/ using document root <none> and password file <none>
I20260808 17:05:41.048688 31048 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:41.048733 31048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:41.048861 31048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:41.049818 31048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "345a79b897ee4482a6be98e9c898c530"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "9ce58014400f6672be4d5dbb47d4cede"
server_key_iv: "d2a8a76cd4b9a20d1c90e54f351023de"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.050137 31048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "345a79b897ee4482a6be98e9c898c530"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "9ce58014400f6672be4d5dbb47d4cede"
server_key_iv: "d2a8a76cd4b9a20d1c90e54f351023de"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.051291 31048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:05:41.051891 31074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.052030 31048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:41.052112 31048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "345a79b897ee4482a6be98e9c898c530"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "9ce58014400f6672be4d5dbb47d4cede"
server_key_iv: "d2a8a76cd4b9a20d1c90e54f351023de"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.052175 31048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:41.066172 31048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:41.068051 31048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:41.068176 31048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:41.068368 31048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:41.068632 31048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:41.068665 31048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.068691 31048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:41.068706 31048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.074060 31048 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:39959
I20260808 17:05:41.074115 31270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:39959 every 8 connection(s)
I20260808 17:05:41.074371 31048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:05:41.076207 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 31048
I20260808 17:05:41.076327 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:05:41.076542 31496 external_mini_cluster.cc:1428] Setting key b6cfaa3e6a254c58946777916dfee4f4
Aug 08 17:05:41 dist-test-slave-2j7r krb5kdc[30638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.2@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:41.083564 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.074948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:41057 (local address 127.30.194.62:46049)
0808 17:05:41.075004 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.075005 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.075188 (+ 183us) server_negotiation.cc:407] Connection header received
0808 17:05:41.075638 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.075639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.075649 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.075690 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:41.076764 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.077353 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.078589 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.078774 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.078835 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.080755 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:41.080757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:41.080757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:41.080764 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:41.082216 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:41.082682 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:41.082683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:41.082684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:41.082744 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:41.082997 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:41.082998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:41.082998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:41.083086 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:41.083129 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.083437 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.083504 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 17:05:41.083854 31275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.074905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:46049 (local address 127.30.194.2:41057)
0808 17:05:41.075069 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.075106 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.075548 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.075753 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.075756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.076402 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:41.076658 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.076664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.077436 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.077438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.078507 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.078510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.078853 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.078855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.078962 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.079349 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:41.079356 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:41.080672 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:41.082298 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:41.082305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:41.082307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:41.082598 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:41.082804 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:41.082805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:41.082805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:41.082938 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:41.083140 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:41.083145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:41.083298 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.083430 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.083585 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260808 17:05:41.084321 31272 heartbeater.cc:347] Connected to a master server at 127.30.194.62:46049
I20260808 17:05:41.084412 31272 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:41.084566 31272 heartbeater.cc:511] Master 127.30.194.62:46049 requested a full tablet report, sending...
I20260808 17:05:41.084921 30728 ts_manager.cc:194] Registered new tserver with Master: 345a79b897ee4482a6be98e9c898c530 (127.30.194.2:39959)
I20260808 17:05:41.085369 30728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.2@KRBTEST.COM'} at 127.30.194.2:41057
WARNING: no policy specified for kudu/127.30.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
Entry for principal kudu/127.30.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab.
I20260808 17:05:41.101195 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46049
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:41.178610 31283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:41.178788 31283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:41.178818 31283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:41.179904 31283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:41.179941 31283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:41.179975 31283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:05:41.181401 31283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.3.keytab
--principal=kudu/127.30.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:46049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:41.181617 31283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:41.181808 31283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:41.186558 31288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.186578 31292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.186599 31290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:41.186697 31283 server_base.cc:1034] running on GCE node
I20260808 17:05:41.186901 31283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:41.187101 31283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:41.188248 31283 hybrid_clock.cc:648] HybridClock initialized: now 1786208741188235 us; error 25 us; skew 500 ppm
Aug 08 17:05:41 dist-test-slave-2j7r krb5kdc[30638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:41.190496 31283 init.cc:377] Logged in from keytab as kudu/127.30.194.3@KRBTEST.COM (short username kudu)
I20260808 17:05:41.190865 31283 webserver.cc:460] Webserver started at http://127.30.194.3:42305/ using document root <none> and password file <none>
I20260808 17:05:41.191061 31283 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:41.191102 31283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:41.191211 31283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:41.192137 31283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "0da9026696ca43bd8a2d655f2be5af57"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "77bf64ce2c31334fa14b82160a6816ea"
server_key_iv: "8e9d7765378a4bf430a5071bb19e0676"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.192458 31283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "0da9026696ca43bd8a2d655f2be5af57"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "77bf64ce2c31334fa14b82160a6816ea"
server_key_iv: "8e9d7765378a4bf430a5071bb19e0676"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.193677 31283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:05:41.194283 31300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.194406 31283 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:41.194483 31283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "0da9026696ca43bd8a2d655f2be5af57"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "77bf64ce2c31334fa14b82160a6816ea"
server_key_iv: "8e9d7765378a4bf430a5071bb19e0676"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.194542 31283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:41.212268 31283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:41.213891 31283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:41.214010 31283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:41.214205 31283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:41.214483 31283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:41.214514 31283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.214534 31283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:41.214557 31283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.220147 31283 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:38637
I20260808 17:05:41.220207 31472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:38637 every 8 connection(s)
I20260808 17:05:41.220480 31283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:05:41.225375 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 31283
I20260808 17:05:41.225476 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:05:41.225667 31496 external_mini_cluster.cc:1428] Setting key 5d954ee4061b19658b61a83c20423cc0
Aug 08 17:05:41 dist-test-slave-2j7r krb5kdc[30638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.3@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:05:41.229163 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.221126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:55133 (local address 127.30.194.62:46049)
0808 17:05:41.221179 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.221181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.221366 (+ 185us) server_negotiation.cc:407] Connection header received
0808 17:05:41.221850 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.221851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.221862 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.221916 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:41.222750 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.223251 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.224281 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.224482 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.224551 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.226875 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:41.226876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:41.226876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:41.226882 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:41.228064 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:41.228452 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:41.228453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:41.228453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:41.228498 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:41.228682 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:41.228682 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:41.228682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:41.228771 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:41.228814 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.229016 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.229070 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:05:41.229374 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.221072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:46049 (local address 127.30.194.3:55133)
0808 17:05:41.221250 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.221283 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.221736 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.221974 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.221978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.222488 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:41.222664 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.222668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.223315 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.223316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.224183 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.224186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.224552 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.224553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.224641 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.225072 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:05:41.225079 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:41.226801 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:41.228142 (+ 1341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:41.228149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:41.228151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:41.228385 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:41.228544 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:05:41.228545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:05:41.228545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:05:41.228636 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:05:41.228815 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:41.228818 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:41.228918 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.228996 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.229160 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260808 17:05:41.229799 31475 heartbeater.cc:347] Connected to a master server at 127.30.194.62:46049
I20260808 17:05:41.229884 31475 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:41.230017 31475 heartbeater.cc:511] Master 127.30.194.62:46049 requested a full tablet report, sending...
I20260808 17:05:41.230329 30728 ts_manager.cc:194] Registered new tserver with Master: 0da9026696ca43bd8a2d655f2be5af57 (127.30.194.3:38637)
I20260808 17:05:41.230698 30728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.194.3@KRBTEST.COM'} at 127.30.194.3:55133
I20260808 17:05:41.237260 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:05:41.243460 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.238517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57754 (local address 127.30.194.62:46049)
0808 17:05:41.238583 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.238585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.238609 (+ 24us) server_negotiation.cc:407] Connection header received
0808 17:05:41.238677 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.238678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.238687 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.238733 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:41.239244 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.239688 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.240378 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.240564 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.240617 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.241671 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:41.241672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:41.241673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:41.241679 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:41.242643 (+ 964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:41.242920 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:41.242921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:41.242921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:41.242961 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:41.243089 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:41.243089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:41.243090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:41.243204 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:41.243278 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.243353 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.243405 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:05:41.244840 30728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:05:41.246387 30728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:05:41.252281 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.248321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:43561 (local address 127.0.0.1:53914)
0808 17:05:41.248569 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.248577 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.248630 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.249183 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.249184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.249187 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:41.249242 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.249244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.250467 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.250468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.251343 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.251346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.252013 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.252014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.252090 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.252103 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.252160 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.252210 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":15,"thread_start_us":70,"threads_started":1}
I20260808 17:05:41.252415 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.248732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.30.194.2:39959)
0808 17:05:41.249019 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.249023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.249030 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:05:41.249061 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.249064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.249078 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.249170 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:41.249347 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.250006 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.251369 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.252088 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.252160 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.252219 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.252300 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.252341 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":86,"threads_started":1}
I20260808 17:05:41.252382 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.248511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:39959 (local address 127.0.0.1:60448)
0808 17:05:41.248732 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.248742 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.248782 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.249246 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.249247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.249249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:41.249276 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.249277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.250093 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.250095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.251290 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.251293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.252166 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.252167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.252237 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.252241 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.252288 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.252322 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":76,"threads_started":1}
I20260808 17:05:41.252457 31492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.248321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:38637 (local address 127.0.0.1:46186)
0808 17:05:41.248477 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.248513 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.248573 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.248867 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.248869 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.248871 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:41.248920 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.248921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.250419 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.250421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.251471 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.251473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.252250 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.252251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.252314 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.252318 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.252356 (+ 38us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.252396 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":22,"thread_start_us":58,"threads_started":1}
I20260808 17:05:41.252708 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.248400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46186 (local address 127.30.194.3:38637)
0808 17:05:41.248708 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.248712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.248722 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:05:41.248755 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.248758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.248772 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.248845 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:41.248978 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.250297 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.251543 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.252162 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.252518 (+ 356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.252578 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.252604 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.252640 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":69,"threads_started":1}
I20260808 17:05:41.252774 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.248434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53914 (local address 127.30.194.1:43561)
0808 17:05:41.248955 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.248959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.248969 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:05:41.249005 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.249007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.249023 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.249178 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:41.249311 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.250357 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.251416 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.251939 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.252003 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.252061 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.252629 (+ 568us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.252673 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":68,"threads_started":1}
I20260808 17:05:41.253373 31377 tablet_service.cc:1467] Processing CreateTablet for tablet e0409ac903c14bde82a1601ae4a07b22 (DEFAULT_TABLE table=test-table [id=fbefdabc87884f6481d73e1021f10119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:41.253383 30926 tablet_service.cc:1467] Processing CreateTablet for tablet e0409ac903c14bde82a1601ae4a07b22 (DEFAULT_TABLE table=test-table [id=fbefdabc87884f6481d73e1021f10119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:41.253568 31162 tablet_service.cc:1467] Processing CreateTablet for tablet e0409ac903c14bde82a1601ae4a07b22 (DEFAULT_TABLE table=test-table [id=fbefdabc87884f6481d73e1021f10119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:41.253719 31377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0409ac903c14bde82a1601ae4a07b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:41.253719 30926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0409ac903c14bde82a1601ae4a07b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:41.253880 31162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0409ac903c14bde82a1601ae4a07b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:41.256268 31505 tablet_bootstrap.cc:492] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: Bootstrap starting.
I20260808 17:05:41.256861 31507 tablet_bootstrap.cc:492] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e: Bootstrap starting.
I20260808 17:05:41.257010 31505 tablet_bootstrap.cc:654] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:41.257573 31507 tablet_bootstrap.cc:654] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:41.257596 31505 log.cc:824] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:41.257931 31510 tablet_bootstrap.cc:492] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: Bootstrap starting.
I20260808 17:05:41.258102 31507 log.cc:824] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:41.258319 31505 tablet_bootstrap.cc:492] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: No bootstrap required, opened a new log
I20260808 17:05:41.258387 31505 ts_tablet_manager.cc:1397] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:05:41.258849 31507 tablet_bootstrap.cc:492] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e: No bootstrap required, opened a new log
I20260808 17:05:41.258916 31507 ts_tablet_manager.cc:1397] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:05:41.260188 31505 raft_consensus.cc:348] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 } } peers { permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 } } peers { permanent_uuid: "345a79b897ee4482a6be98e9c898c530" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39959 } }
I20260808 17:05:41.260363 31505 raft_consensus.cc:374] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:41.260406 31505 raft_consensus.cc:729] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345a79b897ee4482a6be98e9c898c530, State: Initialized, Role: FOLLOWER
I20260808 17:05:41.260521 31505 consensus_queue.cc:260] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 } } peers { permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 } } peers { permanent_uuid: "345a79b897ee4482a6be98e9c898c530" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39959 } }
I20260808 17:05:41.260788 31272 heartbeater.cc:503] Master 127.30.194.62:46049 was elected leader, sending a full tablet report...
I20260808 17:05:41.260807 31505 ts_tablet_manager.cc:1428] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 17:05:41.260746 31507 raft_consensus.cc:348] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 } } peers { permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 } } peers { permanent_uuid: "345a79b897ee4482a6be98e9c898c530" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39959 } }
I20260808 17:05:41.260943 31507 raft_consensus.cc:374] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:41.260993 31507 raft_consensus.cc:729] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5745fa9b7ca84055a1f847ca1870b96e, State: Initialized, Role: FOLLOWER
I20260808 17:05:41.261183 31507 consensus_queue.cc:260] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 } } peers { permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 } } peers { permanent_uuid: "345a79b897ee4482a6be98e9c898c530" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39959 } }
I20260808 17:05:41.261497 31510 tablet_bootstrap.cc:654] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:41.261528 31043 heartbeater.cc:503] Master 127.30.194.62:46049 was elected leader, sending a full tablet report...
I20260808 17:05:41.261515 31507 ts_tablet_manager.cc:1428] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:05:41.262075 31510 log.cc:824] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:41.262733 31510 tablet_bootstrap.cc:492] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: No bootstrap required, opened a new log
I20260808 17:05:41.262789 31510 ts_tablet_manager.cc:1397] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260808 17:05:41.264078 31510 raft_consensus.cc:348] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 } } peers { permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 } } peers { permanent_uuid: "345a79b897ee4482a6be98e9c898c530" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39959 } }
I20260808 17:05:41.264194 31510 raft_consensus.cc:374] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:41.264212 31510 raft_consensus.cc:729] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da9026696ca43bd8a2d655f2be5af57, State: Initialized, Role: FOLLOWER
I20260808 17:05:41.264285 31510 consensus_queue.cc:260] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 } } peers { permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 } } peers { permanent_uuid: "345a79b897ee4482a6be98e9c898c530" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39959 } }
I20260808 17:05:41.264500 31510 ts_tablet_manager.cc:1428] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:05:41.264523 31475 heartbeater.cc:503] Master 127.30.194.62:46049 was elected leader, sending a full tablet report...
I20260808 17:05:41.264722 31514 raft_consensus.cc:482] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:05:41.264787 31514 raft_consensus.cc:504] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 } } peers { permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 } } peers { permanent_uuid: "345a79b897ee4482a6be98e9c898c530" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39959 } }
I20260808 17:05:41.265079 31514 leader_election.cc:288] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da9026696ca43bd8a2d655f2be5af57 (127.30.194.3:38637), 5745fa9b7ca84055a1f847ca1870b96e (127.30.194.1:43561)
I20260808 17:05:41.268168 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.265314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:58619 (local address 127.30.194.3:38637)
0808 17:05:41.265363 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.265365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.265373 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:41.265392 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.265393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.265401 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.265451 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:41.265576 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.266272 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.267530 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.267889 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.267946 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.267953 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.268082 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.268110 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:05:41.268172 31275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.265208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:38637 (local address 127.30.194.2:58619)
0808 17:05:41.265248 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.265259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.265299 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.265455 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.265456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.265458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:41.265511 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.265512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.266339 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.266340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.267438 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.267441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.267953 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.267954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.268027 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.268030 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.268072 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.268111 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 17:05:41.268234 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.265423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:50877 (local address 127.30.194.1:43561)
0808 17:05:41.265455 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.265457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.265570 (+ 113us) server_negotiation.cc:407] Connection header received
0808 17:05:41.265611 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.265612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.265619 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.265666 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:41.265754 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.266412 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.267585 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.267934 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.267994 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.268001 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.268148 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.268179 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:05:41.268266 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.265246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:43561 (local address 127.30.194.2:50877)
0808 17:05:41.265512 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.265522 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.265569 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.265667 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.265668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.265669 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:41.265714 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.265714 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.266468 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.266469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.267527 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.267530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.268010 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.268011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.268091 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.268094 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.268138 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.268174 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":97,"threads_started":1}
I20260808 17:05:41.268494 31405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0409ac903c14bde82a1601ae4a07b22" candidate_uuid: "345a79b897ee4482a6be98e9c898c530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da9026696ca43bd8a2d655f2be5af57" is_pre_election: true
I20260808 17:05:41.268551 30959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0409ac903c14bde82a1601ae4a07b22" candidate_uuid: "345a79b897ee4482a6be98e9c898c530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5745fa9b7ca84055a1f847ca1870b96e" is_pre_election: true
I20260808 17:05:41.268610 31405 raft_consensus.cc:2454] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345a79b897ee4482a6be98e9c898c530 in term 0.
I20260808 17:05:41.268669 30959 raft_consensus.cc:2454] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345a79b897ee4482a6be98e9c898c530 in term 0.
I20260808 17:05:41.268801 31077 leader_election.cc:302] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da9026696ca43bd8a2d655f2be5af57, 345a79b897ee4482a6be98e9c898c530; no voters:
I20260808 17:05:41.268918 31514 raft_consensus.cc:2781] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:05:41.268966 31514 raft_consensus.cc:482] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:05:41.268982 31514 raft_consensus.cc:3037] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:41.269544 31514 raft_consensus.cc:504] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 } } peers { permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 } } peers { permanent_uuid: "345a79b897ee4482a6be98e9c898c530" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39959 } }
I20260808 17:05:41.269640 31514 leader_election.cc:288] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [CANDIDATE]: Term 1 election: Requested vote from peers 0da9026696ca43bd8a2d655f2be5af57 (127.30.194.3:38637), 5745fa9b7ca84055a1f847ca1870b96e (127.30.194.1:43561)
I20260808 17:05:41.269757 30959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0409ac903c14bde82a1601ae4a07b22" candidate_uuid: "345a79b897ee4482a6be98e9c898c530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5745fa9b7ca84055a1f847ca1870b96e"
I20260808 17:05:41.269790 31405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0409ac903c14bde82a1601ae4a07b22" candidate_uuid: "345a79b897ee4482a6be98e9c898c530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da9026696ca43bd8a2d655f2be5af57"
I20260808 17:05:41.269827 30959 raft_consensus.cc:3037] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:41.269862 31405 raft_consensus.cc:3037] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:41.270555 30959 raft_consensus.cc:2454] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345a79b897ee4482a6be98e9c898c530 in term 1.
I20260808 17:05:41.270555 31405 raft_consensus.cc:2454] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345a79b897ee4482a6be98e9c898c530 in term 1.
I20260808 17:05:41.270748 31075 leader_election.cc:302] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345a79b897ee4482a6be98e9c898c530, 5745fa9b7ca84055a1f847ca1870b96e; no voters:
I20260808 17:05:41.270824 31514 raft_consensus.cc:2781] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:41.270949 31514 raft_consensus.cc:686] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 1 LEADER]: Becoming Leader. State: Replica: 345a79b897ee4482a6be98e9c898c530, State: Running, Role: LEADER
I20260808 17:05:41.271008 31514 consensus_queue.cc:237] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 } } peers { permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 } } peers { permanent_uuid: "345a79b897ee4482a6be98e9c898c530" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39959 } }
I20260808 17:05:41.271597 30728 catalog_manager.cc:5495] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 345a79b897ee4482a6be98e9c898c530 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "345a79b897ee4482a6be98e9c898c530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "345a79b897ee4482a6be98e9c898c530" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 39959 } health_report { overall_health: HEALTHY } } }
I20260808 17:05:41.276041 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.273979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.30.194.2:39959)
0808 17:05:41.274012 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.274014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.274021 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:05:41.274089 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.274090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.274104 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.274158 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:05:41.274260 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.274680 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.275385 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.275570 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.275619 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.275755 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:05:41.275848 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:05:41.275911 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.275948 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.275965 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:05:41.277401 30959 raft_consensus.cc:1261] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e [term 1 FOLLOWER]: Refusing update from remote peer 345a79b897ee4482a6be98e9c898c530: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:41.277401 31405 raft_consensus.cc:1261] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57 [term 1 FOLLOWER]: Refusing update from remote peer 345a79b897ee4482a6be98e9c898c530: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:41.277675 31521 consensus_queue.cc:1035] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da9026696ca43bd8a2d655f2be5af57" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:41.277786 31514 consensus_queue.cc:1035] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5745fa9b7ca84055a1f847ca1870b96e" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 43561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:41.279912 31526 mvcc.cc:205] Tried to move back new op lower bound from 7316311004270211072 to 7316311004246417408. Current Snapshot: MvccSnapshot[applied={T|T < 7316311004270211072}]
I20260808 17:05:41.280228 31525 mvcc.cc:205] Tried to move back new op lower bound from 7316311004270211072 to 7316311004246417408. Current Snapshot: MvccSnapshot[applied={T|T < 7316311004270211072}]
I20260808 17:05:41.281829 30725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:41.281903 30725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:41.282498 30725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d73dac3c2c446c6891da44cc64f3a0f: Sending DeleteTablet for 3 replicas of tablet e0409ac903c14bde82a1601ae4a07b22
I20260808 17:05:41.282797 31377 tablet_service.cc:1514] Processing DeleteTablet for tablet e0409ac903c14bde82a1601ae4a07b22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:41 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:46186
I20260808 17:05:41.282804 30926 tablet_service.cc:1514] Processing DeleteTablet for tablet e0409ac903c14bde82a1601ae4a07b22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:41 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:53914
I20260808 17:05:41.282989 31538 tablet_replica.cc:330] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: stopping tablet replica
I20260808 17:05:41.282969 31162 tablet_service.cc:1514] Processing DeleteTablet for tablet e0409ac903c14bde82a1601ae4a07b22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:41 UTC) from {username='kudu', principal='kudu/127.30.194.62@KRBTEST.COM'} at 127.0.0.1:60448
I20260808 17:05:41.283099 31538 raft_consensus.cc:2229] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:41.283109 31539 tablet_replica.cc:330] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e: stopping tablet replica
I20260808 17:05:41.283217 31539 raft_consensus.cc:2229] T e0409ac903c14bde82a1601ae4a07b22 P 5745fa9b7ca84055a1f847ca1870b96e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:41.283239 31538 raft_consensus.cc:2258] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:41.283427 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 30820
I20260808 17:05:41.283600 31538 ts_tablet_manager.cc:1905] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:41.285205 31538 ts_tablet_manager.cc:1918] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:41.285319 31538 log.cc:1196] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/e0409ac903c14bde82a1601ae4a07b22
I20260808 17:05:41.285625 31538 ts_tablet_manager.cc:1939] T e0409ac903c14bde82a1601ae4a07b22 P 0da9026696ca43bd8a2d655f2be5af57: Deleting consensus metadata
I20260808 17:05:41.286077 30702 catalog_manager.cc:4841] TS 0da9026696ca43bd8a2d655f2be5af57 (127.30.194.3:38637): tablet e0409ac903c14bde82a1601ae4a07b22 (table test-table [id=fbefdabc87884f6481d73e1021f10119]) successfully deleted
I20260808 17:05:41.288450 31541 tablet_replica.cc:330] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: stopping tablet replica
I20260808 17:05:41.288543 31541 raft_consensus.cc:2229] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:05:41.288641 31541 raft_consensus.cc:2258] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:41.289039 31541 ts_tablet_manager.cc:1905] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:41.290486 31541 ts_tablet_manager.cc:1918] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:41.290556 31541 log.cc:1196] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/e0409ac903c14bde82a1601ae4a07b22
I20260808 17:05:41.290869 31541 ts_tablet_manager.cc:1939] T e0409ac903c14bde82a1601ae4a07b22 P 345a79b897ee4482a6be98e9c898c530: Deleting consensus metadata
I20260808 17:05:41.291276 30702 catalog_manager.cc:4841] TS 345a79b897ee4482a6be98e9c898c530 (127.30.194.2:39959): tablet e0409ac903c14bde82a1601ae4a07b22 (table test-table [id=fbefdabc87884f6481d73e1021f10119]) successfully deleted
W20260808 17:05:41.292711 30696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:43561 (error 108)
W20260808 17:05:41.293583 31502 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:05:41.293061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:43561 (local address 127.0.0.1:53930)
0808 17:05:41.293134 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.293146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.293190 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.293539 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:43561: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 17:05:41.293712 30696 catalog_manager.cc:4584] TS 5745fa9b7ca84055a1f847ca1870b96e (127.30.194.1:43561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e0409ac903c14bde82a1601ae4a07b22: Network error: Client connection negotiation failed: client connection to 127.30.194.1:43561: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 17:05:41.293761 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 31048
I20260808 17:05:41.303936 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 31283
I20260808 17:05:41.314960 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 30671
2026-08-08T17:05:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1995 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T17:05:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:41Z Disabled control of system clock
I20260808 17:05:41.350410 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:33001
--webserver_interface=127.30.194.62
--webserver_port=0
--builtin_ntp_servers=127.30.194.20:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:33001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:05:41.431147 31548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:41.431357 31548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:41.431376 31548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:41.432487 31548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:41.432524 31548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:41.432536 31548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:41.432546 31548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:41.432564 31548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:41.433867 31548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:33001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:33001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:41.434055 31548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:41.434283 31548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:41.438617 31556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.438647 31558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.438632 31554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:41.438956 31548 server_base.cc:1034] running on GCE node
I20260808 17:05:41.439185 31548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:41.439465 31548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:41.440621 31548 hybrid_clock.cc:648] HybridClock initialized: now 1786208741440606 us; error 32 us; skew 500 ppm
I20260808 17:05:41.441803 31548 webserver.cc:460] Webserver started at http://127.30.194.62:35373/ using document root <none> and password file <none>
I20260808 17:05:41.442014 31548 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:41.442058 31548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:41.442196 31548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:41.443225 31548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "4612cc3227c444e59ba845ef9a1a9f01"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "ca4fe4d832554c095304c62f082cd67f"
server_key_iv: "2aa473ef93dbdf0542fb33909fe20efe"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.443542 31548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "4612cc3227c444e59ba845ef9a1a9f01"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "ca4fe4d832554c095304c62f082cd67f"
server_key_iv: "2aa473ef93dbdf0542fb33909fe20efe"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.444726 31548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:05:41.445415 31567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.445547 31548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.445626 31548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "4612cc3227c444e59ba845ef9a1a9f01"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "ca4fe4d832554c095304c62f082cd67f"
server_key_iv: "2aa473ef93dbdf0542fb33909fe20efe"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.445690 31548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:41.471151 31548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:41.471467 31548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:41.471594 31548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:41.475274 31548 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:33001
I20260808 17:05:41.475309 31653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:33001 every 8 connection(s)
I20260808 17:05:41.475612 31548 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:41.476511 31655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:41.478710 31655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01: Bootstrap starting.
I20260808 17:05:41.479254 31655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:41.479519 31655 log.cc:824] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:41.480129 31655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01: No bootstrap required, opened a new log
I20260808 17:05:41.481639 31655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612cc3227c444e59ba845ef9a1a9f01" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33001 } }
I20260808 17:05:41.481775 31655 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:41.481796 31655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4612cc3227c444e59ba845ef9a1a9f01, State: Initialized, Role: FOLLOWER
I20260808 17:05:41.481938 31655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612cc3227c444e59ba845ef9a1a9f01" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33001 } }
I20260808 17:05:41.482000 31655 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:41.482023 31655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:41.482066 31655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:41.482542 31655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612cc3227c444e59ba845ef9a1a9f01" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33001 } }
I20260808 17:05:41.482631 31655 leader_election.cc:302] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4612cc3227c444e59ba845ef9a1a9f01; no voters:
I20260808 17:05:41.482652 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 31548
I20260808 17:05:41.482759 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:41.482779 31655 leader_election.cc:288] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:41.482846 31658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:41.482959 31496 external_mini_cluster.cc:1428] Setting key e065cef2187f6623792eec052206fc55
I20260808 17:05:41.483023 31655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:41.483062 31658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [term 1 LEADER]: Becoming Leader. State: Replica: 4612cc3227c444e59ba845ef9a1a9f01, State: Running, Role: LEADER
I20260808 17:05:41.483172 31658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612cc3227c444e59ba845ef9a1a9f01" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33001 } }
I20260808 17:05:41.483527 31658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4612cc3227c444e59ba845ef9a1a9f01. Latest consensus state: current_term: 1 leader_uuid: "4612cc3227c444e59ba845ef9a1a9f01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612cc3227c444e59ba845ef9a1a9f01" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33001 } } }
I20260808 17:05:41.483605 31658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:41.483767 31661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4612cc3227c444e59ba845ef9a1a9f01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4612cc3227c444e59ba845ef9a1a9f01" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33001 } } }
I20260808 17:05:41.483844 31661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01 [sys.catalog]: This master's current role is: LEADER
W20260808 17:05:41.484699 31683 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:05:41.484755 31683 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:05:41.484803 31672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:41.484970 31672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:41.486536 31672 catalog_manager.cc:1347] Generated new cluster ID: 5bcced8dcda94a52803fcb698830912d
I20260808 17:05:41.486588 31672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:41.488924 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.483427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.30.194.62:33001)
0808 17:05:41.483776 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.483782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.483806 (+ 24us) server_negotiation.cc:407] Connection header received
0808 17:05:41.483911 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.483918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.483982 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.484115 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:41.484451 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.485888 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.486923 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.487539 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.487674 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.488237 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:41.488242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:41.488242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:41.488452 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:41.488494 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:41.488509 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:41.488580 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.488639 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.488716 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":60,"threads_started":1}
I20260808 17:05:41.491202 31672 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:41.491626 31672 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:41.499660 31672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01: Generated new TSK 0
I20260808 17:05:41.499806 31672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:41.501179 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.1:0
--local_ip_for_outbound_sockets=127.30.194.1
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33001
--builtin_ntp_servers=127.30.194.20:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:05:41.580760 31691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:41.580945 31691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:41.580971 31691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:41.582089 31691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:41.582129 31691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:41.582159 31691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.1
I20260808 17:05:41.583541 31691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.194.1
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.1
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:41.583756 31691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:41.584030 31691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:41.588605 31699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.588641 31704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:41.589119 31691 server_base.cc:1034] running on GCE node
W20260808 17:05:41.588644 31700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:41.589828 31691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:41.590044 31691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:41.591192 31691 hybrid_clock.cc:648] HybridClock initialized: now 1786208741591180 us; error 29 us; skew 500 ppm
I20260808 17:05:41.592410 31691 webserver.cc:460] Webserver started at http://127.30.194.1:40267/ using document root <none> and password file <none>
I20260808 17:05:41.592623 31691 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:41.592669 31691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:41.592800 31691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:41.593829 31691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/data/instance:
uuid: "c01de2a01e2c4d2ea7ba62295ae838c9"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "47996d350b4cf2dbe7967762ca804055"
server_key_iv: "70e09fb11ef2da7dbb75bdbcca423b9c"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.594137 31691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance:
uuid: "c01de2a01e2c4d2ea7ba62295ae838c9"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "47996d350b4cf2dbe7967762ca804055"
server_key_iv: "70e09fb11ef2da7dbb75bdbcca423b9c"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.595327 31691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:05:41.595988 31717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.596148 31691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.596267 31691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/wal
uuid: "c01de2a01e2c4d2ea7ba62295ae838c9"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "47996d350b4cf2dbe7967762ca804055"
server_key_iv: "70e09fb11ef2da7dbb75bdbcca423b9c"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.596342 31691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:41.608117 31691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:41.608348 31691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:41.608443 31691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:41.608609 31691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:41.608856 31691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:41.608886 31691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.608912 31691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:41.608927 31691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.614082 31691 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.1:35231
I20260808 17:05:41.614146 31920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.1:35231 every 8 connection(s)
I20260808 17:05:41.614421 31691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/data/info.pb
I20260808 17:05:41.616276 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 31691
I20260808 17:05:41.616405 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-0/wal/instance
I20260808 17:05:41.616609 31496 external_mini_cluster.cc:1428] Setting key 6db3471f2166d8f1cdbc5d48e0aa6a7f
I20260808 17:05:41.617606 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.2:0
--local_ip_for_outbound_sockets=127.30.194.2
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33001
--builtin_ntp_servers=127.30.194.20:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:05:41.619601 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.615112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.1:50285 (local address 127.30.194.62:33001)
0808 17:05:41.615173 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.615176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.615375 (+ 199us) server_negotiation.cc:407] Connection header received
0808 17:05:41.615808 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.615809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.615821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.615872 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:41.616659 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.617277 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.618262 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.618475 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.618529 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.619285 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:41.619287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:41.619287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:41.619291 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:41.619307 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:41.619311 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:41.619357 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.619470 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.619534 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 17:05:41.619764 31929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.615070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:33001 (local address 127.30.194.1:50285)
0808 17:05:41.615262 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.615296 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.615731 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.615933 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.615936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.615944 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:41.616573 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.616580 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.617362 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.617363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.618184 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.618187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.618544 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.618545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.618669 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.619125 (+ 456us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:05:41.619126 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:41.619165 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:05:41.619194 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:41.619368 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:41.619372 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:41.619375 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.619465 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.619563 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260808 17:05:41.620188 31921 heartbeater.cc:347] Connected to a master server at 127.30.194.62:33001
I20260808 17:05:41.620286 31921 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:41.620455 31921 heartbeater.cc:511] Master 127.30.194.62:33001 requested a full tablet report, sending...
I20260808 17:05:41.620904 31600 ts_manager.cc:194] Registered new tserver with Master: c01de2a01e2c4d2ea7ba62295ae838c9 (127.30.194.1:35231)
I20260808 17:05:41.621515 31600 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.1:50285
W20260808 17:05:41.698763 31930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:41.698951 31930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:41.698978 31930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:41.700158 31930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:41.700198 31930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:41.700240 31930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.2
I20260808 17:05:41.701640 31930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.194.2
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.2
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:41.701853 31930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:41.702103 31930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:41.706641 31941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.706678 31938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.706727 31939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:41.706888 31930 server_base.cc:1034] running on GCE node
I20260808 17:05:41.707053 31930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:41.707233 31930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:41.708353 31930 hybrid_clock.cc:648] HybridClock initialized: now 1786208741708347 us; error 26 us; skew 500 ppm
I20260808 17:05:41.709481 31930 webserver.cc:460] Webserver started at http://127.30.194.2:35661/ using document root <none> and password file <none>
I20260808 17:05:41.709671 31930 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:41.709713 31930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:41.709815 31930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:41.710753 31930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/data/instance:
uuid: "60bc252269a34b13a790ac05e06983b7"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "3fc9fcf37a6450c2747a0277d2fe0ef9"
server_key_iv: "7c8c2daa571a753752fe29f566ef2f5c"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.711066 31930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance:
uuid: "60bc252269a34b13a790ac05e06983b7"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "3fc9fcf37a6450c2747a0277d2fe0ef9"
server_key_iv: "7c8c2daa571a753752fe29f566ef2f5c"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.712216 31930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:05:41.712870 31950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.713020 31930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:41.713124 31930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/wal
uuid: "60bc252269a34b13a790ac05e06983b7"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "3fc9fcf37a6450c2747a0277d2fe0ef9"
server_key_iv: "7c8c2daa571a753752fe29f566ef2f5c"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.713196 31930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:41.723775 31930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:41.723996 31930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:41.724097 31930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:41.724275 31930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:41.724526 31930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:41.724557 31930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.724584 31930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:41.724603 31930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.730382 31930 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.2:36743
I20260808 17:05:41.730417 32139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.2:36743 every 8 connection(s)
I20260808 17:05:41.730700 31930 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/data/info.pb
I20260808 17:05:41.733582 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 31930
I20260808 17:05:41.733698 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/wal/instance
I20260808 17:05:41.733883 31496 external_mini_cluster.cc:1428] Setting key 15e3d6d9504e7ae85e50285df8d424d3
I20260808 17:05:41.734790 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.194.3:0
--local_ip_for_outbound_sockets=127.30.194.3
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33001
--builtin_ntp_servers=127.30.194.20:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:05:41.735404 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.731303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:36863 (local address 127.30.194.62:33001)
0808 17:05:41.731361 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.731363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.731570 (+ 207us) server_negotiation.cc:407] Connection header received
0808 17:05:41.731988 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.731989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.731999 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.732046 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:41.732596 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.733079 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.734111 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.734342 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.734404 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.735142 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:41.735143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:41.735144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:41.735149 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:41.735167 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:41.735173 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:41.735243 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.735294 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.735324 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:05:41.735525 32143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.731280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:33001 (local address 127.30.194.2:36863)
0808 17:05:41.731465 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.731501 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.731895 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.732092 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.732095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.732102 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:41.732534 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.732538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.733186 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.733187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.734029 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.734033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.734417 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.734419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.734549 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.735001 (+ 452us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:05:41.735001 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:41.735036 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:05:41.735061 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:41.735243 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:41.735246 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:41.735249 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.735291 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.735363 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260808 17:05:41.736721 32141 heartbeater.cc:347] Connected to a master server at 127.30.194.62:33001
I20260808 17:05:41.736810 32141 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:41.736974 32141 heartbeater.cc:511] Master 127.30.194.62:33001 requested a full tablet report, sending...
I20260808 17:05:41.737334 31600 ts_manager.cc:194] Registered new tserver with Master: 60bc252269a34b13a790ac05e06983b7 (127.30.194.2:36743)
I20260808 17:05:41.737759 31600 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.2:36863
W20260808 17:05:41.813722 32148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:41.813894 32148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:41.813915 32148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:41.814960 32148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:41.814993 32148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:41.815017 32148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.194.3
I20260808 17:05:41.816367 32148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.194.3
--webserver_port=0
--tserver_master_addrs=127.30.194.62:33001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.194.3
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:41.816547 32148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:41.816778 32148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:41.821273 32160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.821319 32158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:41.822743 32165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:41.823580 32148 server_base.cc:1034] running on GCE node
I20260808 17:05:41.823779 32148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:41.823999 32148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:41.825135 32148 hybrid_clock.cc:648] HybridClock initialized: now 1786208741825113 us; error 34 us; skew 500 ppm
I20260808 17:05:41.826244 32148 webserver.cc:460] Webserver started at http://127.30.194.3:43893/ using document root <none> and password file <none>
I20260808 17:05:41.826483 32148 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:41.826531 32148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:41.826654 32148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:41.827591 32148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/data/instance:
uuid: "e48e7c8d90c04f56949a5217ef950659"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "bcb0f558c3851b061af0c4f03e27cbe0"
server_key_iv: "19ec565df6f202d485a3b007905caf4e"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.827906 32148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance:
uuid: "e48e7c8d90c04f56949a5217ef950659"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "bcb0f558c3851b061af0c4f03e27cbe0"
server_key_iv: "19ec565df6f202d485a3b007905caf4e"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.829044 32148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:05:41.829770 32173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.829921 32148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:41.830009 32148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/wal
uuid: "e48e7c8d90c04f56949a5217ef950659"
format_stamp: "Formatted at 2026-08-08 17:05:41 on dist-test-slave-2j7r"
server_key: "bcb0f558c3851b061af0c4f03e27cbe0"
server_key_iv: "19ec565df6f202d485a3b007905caf4e"
server_key_version: "encryptionkey@0"
I20260808 17:05:41.830075 32148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:41.861743 32148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:41.862048 32148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:41.862175 32148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:41.862414 32148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:05:41.862735 32148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:05:41.862769 32148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.862798 32148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:05:41.862816 32148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:41.868395 32148 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.3:38871
I20260808 17:05:41.868451 32358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.3:38871 every 8 connection(s)
I20260808 17:05:41.868748 32148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/data/info.pb
I20260808 17:05:41.870430 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 32148
I20260808 17:05:41.870532 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/wal/instance
I20260808 17:05:41.870704 31496 external_mini_cluster.cc:1428] Setting key 969adf72e9af312c30daeeda140de1ca
I20260808 17:05:41.873962 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.869465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.3:52155 (local address 127.30.194.62:33001)
0808 17:05:41.869536 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.869538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.869767 (+ 229us) server_negotiation.cc:407] Connection header received
0808 17:05:41.870300 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.870302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.870317 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.870378 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:41.871200 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.871851 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.872809 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.872990 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.873031 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.873744 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:41.873745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:41.873746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:41.873750 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:41.873765 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:41.873769 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:41.873815 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.873880 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.873903 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:05:41.874171 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.869425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:33001 (local address 127.30.194.3:52155)
0808 17:05:41.869631 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.869664 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.870190 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.870452 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.870456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.870467 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:05:41.871093 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.871100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.871934 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.871936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.872731 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.872733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.873047 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.873047 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.873175 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.873603 (+ 428us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:05:41.873604 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:05:41.873632 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:05:41.873656 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:05:41.873823 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:05:41.873826 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:05:41.873828 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.873879 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.873956 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260808 17:05:41.874639 32359 heartbeater.cc:347] Connected to a master server at 127.30.194.62:33001
I20260808 17:05:41.874737 32359 heartbeater.cc:464] Registering TS with master...
I20260808 17:05:41.874913 32359 heartbeater.cc:511] Master 127.30.194.62:33001 requested a full tablet report, sending...
I20260808 17:05:41.875288 31600 ts_manager.cc:194] Registered new tserver with Master: e48e7c8d90c04f56949a5217ef950659 (127.30.194.3:38871)
I20260808 17:05:41.875643 31600 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.194.3:52155
I20260808 17:05:41.882215 31496 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:05:41.885911 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.883487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52204 (local address 127.30.194.62:33001)
0808 17:05:41.883533 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.883534 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.883652 (+ 118us) server_negotiation.cc:407] Connection header received
0808 17:05:41.883694 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.883694 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.883704 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.883748 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:41.883912 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.884344 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.885028 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.885237 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.885279 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.885749 (+ 470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:41.885749 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:41.885750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:41.885753 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:41.885764 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:41.885767 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:41.885806 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.885843 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.885862 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:05:41.887178 31600 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:05:41.888754 31600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:05:41.895401 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.891552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:35231 (local address 127.0.0.1:39270)
0808 17:05:41.891836 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.891866 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.891927 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.892098 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.892102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.892104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:41.892161 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.892163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.893421 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.893422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.894325 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.894328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.895115 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.895116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.895190 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.895203 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.895279 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.895327 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":36,"thread_start_us":74,"threads_started":1}
I20260808 17:05:41.895411 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.891659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39270 (local address 127.30.194.1:35231)
0808 17:05:41.891832 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.891837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.891914 (+ 77us) server_negotiation.cc:407] Connection header received
0808 17:05:41.891998 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.892000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.892014 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.892077 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:41.892233 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.893332 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.894411 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.895036 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.895106 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.895189 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.895294 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.895333 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260808 17:05:41.895831 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.891667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.2:36743 (local address 127.0.0.1:33826)
0808 17:05:41.892144 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.892153 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.892206 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.892454 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.892455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.892457 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:41.892494 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.892495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.893687 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.893688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.894725 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.894727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.895603 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.895604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.895657 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.895660 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.895731 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.895769 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":90,"threads_started":1}
I20260808 17:05:41.895882 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.891889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33826 (local address 127.30.194.2:36743)
0808 17:05:41.892072 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.892076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.892234 (+ 158us) server_negotiation.cc:407] Connection header received
0808 17:05:41.892282 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.892284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.892317 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.892389 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:41.892571 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.893604 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.894824 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.895421 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.895483 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.895571 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.895757 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.895803 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260808 17:05:41.896095 31794 tablet_service.cc:1467] Processing CreateTablet for tablet 6959f33bec39475f85f9927c6eb20ce3 (DEFAULT_TABLE table=test-table [id=34f39ac5d38d48ffbeff2eccafbd3edc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:41.896389 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.891496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:38871 (local address 127.0.0.1:44558)
0808 17:05:41.891823 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:41.891861 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:41.891920 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:41.892605 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:41.892606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:41.892608 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:41.892655 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.892657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.893551 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.893552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.894676 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:41.894679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.896160 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.896161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:41.896242 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.896246 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:05:41.896297 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:05:41.896335 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":47,"threads_started":1}
I20260808 17:05:41.896448 31794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6959f33bec39475f85f9927c6eb20ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:41.896586 32021 tablet_service.cc:1467] Processing CreateTablet for tablet 6959f33bec39475f85f9927c6eb20ce3 (DEFAULT_TABLE table=test-table [id=34f39ac5d38d48ffbeff2eccafbd3edc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:41.896723 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:41.891556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.30.194.3:38871)
0808 17:05:41.892304 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:41.892308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:41.892316 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:41.892349 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:41.892461 (+ 112us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:41.892480 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:41.892550 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:41.892720 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.893475 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.894757 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:41.895420 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:41.895771 (+ 351us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:41.895855 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:41.896609 (+ 754us) server_negotiation.cc:299] Negotiation successful
0808 17:05:41.896647 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":73,"threads_started":1}
I20260808 17:05:41.896910 32021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6959f33bec39475f85f9927c6eb20ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:41.897554 32251 tablet_service.cc:1467] Processing CreateTablet for tablet 6959f33bec39475f85f9927c6eb20ce3 (DEFAULT_TABLE table=test-table [id=34f39ac5d38d48ffbeff2eccafbd3edc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:05:41.897825 32251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6959f33bec39475f85f9927c6eb20ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:41.899786 32385 tablet_bootstrap.cc:492] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9: Bootstrap starting.
I20260808 17:05:41.900132 32387 tablet_bootstrap.cc:492] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: Bootstrap starting.
I20260808 17:05:41.900527 32385 tablet_bootstrap.cc:654] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:41.900705 32387 tablet_bootstrap.cc:654] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:41.900813 32385 log.cc:824] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:41.900990 32387 log.cc:824] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:41.901225 32390 tablet_bootstrap.cc:492] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: Bootstrap starting.
I20260808 17:05:41.901508 32385 tablet_bootstrap.cc:492] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9: No bootstrap required, opened a new log
I20260808 17:05:41.901564 32385 ts_tablet_manager.cc:1397] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:05:41.901732 32390 tablet_bootstrap.cc:654] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:41.901983 32390 log.cc:824] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:41.902599 32390 tablet_bootstrap.cc:492] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: No bootstrap required, opened a new log
I20260808 17:05:41.902649 32390 ts_tablet_manager.cc:1397] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:05:41.903463 32385 raft_consensus.cc:348] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 } } peers { permanent_uuid: "60bc252269a34b13a790ac05e06983b7" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36743 } } peers { permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 } }
I20260808 17:05:41.903585 32385 raft_consensus.cc:374] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:41.903615 32385 raft_consensus.cc:729] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01de2a01e2c4d2ea7ba62295ae838c9, State: Initialized, Role: FOLLOWER
I20260808 17:05:41.903731 32385 consensus_queue.cc:260] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 } } peers { permanent_uuid: "60bc252269a34b13a790ac05e06983b7" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36743 } } peers { permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 } }
I20260808 17:05:41.903971 32385 ts_tablet_manager.cc:1428] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:05:41.904014 31921 heartbeater.cc:503] Master 127.30.194.62:33001 was elected leader, sending a full tablet report...
I20260808 17:05:41.904650 32387 tablet_bootstrap.cc:492] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: No bootstrap required, opened a new log
I20260808 17:05:41.904719 32387 ts_tablet_manager.cc:1397] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 17:05:41.904814 32390 raft_consensus.cc:348] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 } } peers { permanent_uuid: "60bc252269a34b13a790ac05e06983b7" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36743 } } peers { permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 } }
I20260808 17:05:41.904954 32390 raft_consensus.cc:374] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:41.904989 32390 raft_consensus.cc:729] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48e7c8d90c04f56949a5217ef950659, State: Initialized, Role: FOLLOWER
I20260808 17:05:41.905117 32390 consensus_queue.cc:260] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 } } peers { permanent_uuid: "60bc252269a34b13a790ac05e06983b7" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36743 } } peers { permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 } }
I20260808 17:05:41.905375 32390 ts_tablet_manager.cc:1428] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:05:41.905424 32359 heartbeater.cc:503] Master 127.30.194.62:33001 was elected leader, sending a full tablet report...
I20260808 17:05:41.906648 32387 raft_consensus.cc:348] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 } } peers { permanent_uuid: "60bc252269a34b13a790ac05e06983b7" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36743 } } peers { permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 } }
I20260808 17:05:41.906761 32387 raft_consensus.cc:374] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:41.906782 32387 raft_consensus.cc:729] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60bc252269a34b13a790ac05e06983b7, State: Initialized, Role: FOLLOWER
I20260808 17:05:41.906884 32387 consensus_queue.cc:260] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 } } peers { permanent_uuid: "60bc252269a34b13a790ac05e06983b7" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36743 } } peers { permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 } }
I20260808 17:05:41.907104 32387 ts_tablet_manager.cc:1428] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260808 17:05:41.907150 32141 heartbeater.cc:503] Master 127.30.194.62:33001 was elected leader, sending a full tablet report...
W20260808 17:05:41.981220 32142 tablet.cc:2369] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:05:42.115065 31924 tablet.cc:2369] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:05:42.119228 32361 tablet.cc:2369] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:05:42.134760 32401 raft_consensus.cc:482] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:05:42.134876 32401 raft_consensus.cc:504] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 } } peers { permanent_uuid: "60bc252269a34b13a790ac05e06983b7" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36743 } } peers { permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 } }
I20260808 17:05:42.135217 32401 leader_election.cc:288] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e48e7c8d90c04f56949a5217ef950659 (127.30.194.3:38871), c01de2a01e2c4d2ea7ba62295ae838c9 (127.30.194.1:35231)
I20260808 17:05:42.137948 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:42.135470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:48441 (local address 127.30.194.3:38871)
0808 17:05:42.135513 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:42.135514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:42.135523 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:05:42.135580 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:42.135581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:42.135591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:42.135651 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:42.135798 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:42.136298 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:42.137275 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:42.137632 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:42.137684 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:42.137692 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:42.137837 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 17:05:42.137878 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:05:42.137960 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:42.135460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.194.2:53451 (local address 127.30.194.1:35231)
0808 17:05:42.135504 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:42.135506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:42.135654 (+ 148us) server_negotiation.cc:407] Connection header received
0808 17:05:42.135686 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:42.135687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:42.135699 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:42.135750 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:05:42.135901 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:42.136351 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:42.137299 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:42.137658 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:42.137723 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:42.137731 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:42.137861 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 17:05:42.137897 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:05:42.137969 32143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:42.135392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.3:38871 (local address 127.30.194.2:48441)
0808 17:05:42.135453 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:42.135467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:42.135514 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:42.135663 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:42.135664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:42.135666 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:42.135735 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:42.135738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:42.136370 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:42.136371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:42.137199 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:42.137201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:42.137705 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:42.137706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:42.137769 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:42.137772 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:42.137818 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 17:05:42.137865 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":28}
I20260808 17:05:42.137980 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:42.135396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:35231 (local address 127.30.194.2:53451)
0808 17:05:42.135575 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:42.135585 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:05:42.135646 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:05:42.135764 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:05:42.135765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:05:42.135766 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:05:42.135841 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:42.135842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:42.136422 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:42.136423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:42.137210 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:05:42.137212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:05:42.137734 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:05:42.137735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:05:42.137795 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:42.137798 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:05:42.137848 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:05:42.137882 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":13,"thread_start_us":103,"threads_started":1}
I20260808 17:05:42.138267 32286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6959f33bec39475f85f9927c6eb20ce3" candidate_uuid: "60bc252269a34b13a790ac05e06983b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e7c8d90c04f56949a5217ef950659" is_pre_election: true
I20260808 17:05:42.138414 32286 raft_consensus.cc:2454] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60bc252269a34b13a790ac05e06983b7 in term 0.
I20260808 17:05:42.138396 31832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6959f33bec39475f85f9927c6eb20ce3" candidate_uuid: "60bc252269a34b13a790ac05e06983b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" is_pre_election: true
I20260808 17:05:42.138528 31832 raft_consensus.cc:2454] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60bc252269a34b13a790ac05e06983b7 in term 0.
I20260808 17:05:42.138616 31955 leader_election.cc:302] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60bc252269a34b13a790ac05e06983b7, e48e7c8d90c04f56949a5217ef950659; no voters:
I20260808 17:05:42.138767 32401 raft_consensus.cc:2781] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:05:42.138819 32401 raft_consensus.cc:482] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:05:42.138841 32401 raft_consensus.cc:3037] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:42.139541 32401 raft_consensus.cc:504] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 } } peers { permanent_uuid: "60bc252269a34b13a790ac05e06983b7" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36743 } } peers { permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 } }
I20260808 17:05:42.139657 32401 leader_election.cc:288] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [CANDIDATE]: Term 1 election: Requested vote from peers e48e7c8d90c04f56949a5217ef950659 (127.30.194.3:38871), c01de2a01e2c4d2ea7ba62295ae838c9 (127.30.194.1:35231)
I20260808 17:05:42.139796 31832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6959f33bec39475f85f9927c6eb20ce3" candidate_uuid: "60bc252269a34b13a790ac05e06983b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9"
I20260808 17:05:42.139858 31832 raft_consensus.cc:3037] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:42.139855 32286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6959f33bec39475f85f9927c6eb20ce3" candidate_uuid: "60bc252269a34b13a790ac05e06983b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e7c8d90c04f56949a5217ef950659"
I20260808 17:05:42.139925 32286 raft_consensus.cc:3037] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:42.140554 31832 raft_consensus.cc:2454] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60bc252269a34b13a790ac05e06983b7 in term 1.
I20260808 17:05:42.140606 32286 raft_consensus.cc:2454] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60bc252269a34b13a790ac05e06983b7 in term 1.
I20260808 17:05:42.140729 31954 leader_election.cc:302] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60bc252269a34b13a790ac05e06983b7, c01de2a01e2c4d2ea7ba62295ae838c9; no voters:
I20260808 17:05:42.140844 32401 raft_consensus.cc:2781] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:42.140960 32401 raft_consensus.cc:686] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 1 LEADER]: Becoming Leader. State: Replica: 60bc252269a34b13a790ac05e06983b7, State: Running, Role: LEADER
I20260808 17:05:42.141047 32401 consensus_queue.cc:237] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 } } peers { permanent_uuid: "60bc252269a34b13a790ac05e06983b7" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36743 } } peers { permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 } }
I20260808 17:05:42.141789 31600 catalog_manager.cc:5495] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60bc252269a34b13a790ac05e06983b7 (127.30.194.2). New cstate: current_term: 1 leader_uuid: "60bc252269a34b13a790ac05e06983b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60bc252269a34b13a790ac05e06983b7" member_type: VOTER last_known_addr { host: "127.30.194.2" port: 36743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 } health_report { overall_health: UNKNOWN } } }
I20260808 17:05:42.175951 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:42.174000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33828 (local address 127.30.194.2:36743)
0808 17:05:42.174050 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:42.174051 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:42.174059 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:42.174113 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:42.174114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:42.174129 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:42.174184 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:05:42.174283 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:42.174720 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:42.175363 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:42.175529 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:42.175569 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:42.175669 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:05:42.175766 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:05:42.175820 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:42.175850 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 17:05:42.175863 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:05:42.177577 32286 raft_consensus.cc:1261] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659 [term 1 FOLLOWER]: Refusing update from remote peer 60bc252269a34b13a790ac05e06983b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:42.177598 31832 raft_consensus.cc:1261] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9 [term 1 FOLLOWER]: Refusing update from remote peer 60bc252269a34b13a790ac05e06983b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:05:42.177914 32401 consensus_queue.cc:1035] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01de2a01e2c4d2ea7ba62295ae838c9" member_type: VOTER last_known_addr { host: "127.30.194.1" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:42.178052 32404 consensus_queue.cc:1035] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48e7c8d90c04f56949a5217ef950659" member_type: VOTER last_known_addr { host: "127.30.194.3" port: 38871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:05:42.180116 32407 mvcc.cc:205] Tried to move back new op lower bound from 7316311007956865024 to 7316311007810232320. Current Snapshot: MvccSnapshot[applied={T|T < 7316311007956865024}]
I20260808 17:05:42.180883 32409 mvcc.cc:205] Tried to move back new op lower bound from 7316311007956865024 to 7316311007810232320. Current Snapshot: MvccSnapshot[applied={T|T < 7316311007956865024}]
I20260808 17:05:42.181282 32408 mvcc.cc:205] Tried to move back new op lower bound from 7316311007956865024 to 7316311007810232320. Current Snapshot: MvccSnapshot[applied={T|T < 7316311007956865024}]
I20260808 17:05:42.182089 31600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:42.182169 31600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:05:42.182834 31600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4612cc3227c444e59ba845ef9a1a9f01: Sending DeleteTablet for 3 replicas of tablet 6959f33bec39475f85f9927c6eb20ce3
I20260808 17:05:42.183161 32251 tablet_service.cc:1514] Processing DeleteTablet for tablet 6959f33bec39475f85f9927c6eb20ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:42 UTC) from {username='slave'} at 127.0.0.1:44558
I20260808 17:05:42.183235 31794 tablet_service.cc:1514] Processing DeleteTablet for tablet 6959f33bec39475f85f9927c6eb20ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:42 UTC) from {username='slave'} at 127.0.0.1:39270
I20260808 17:05:42.183470 32421 tablet_replica.cc:330] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9: stopping tablet replica
I20260808 17:05:42.183465 32021 tablet_service.cc:1514] Processing DeleteTablet for tablet 6959f33bec39475f85f9927c6eb20ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:05:42 UTC) from {username='slave'} at 127.0.0.1:33826
I20260808 17:05:42.183568 32421 raft_consensus.cc:2229] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:42.183691 32421 raft_consensus.cc:2258] T 6959f33bec39475f85f9927c6eb20ce3 P c01de2a01e2c4d2ea7ba62295ae838c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:42.183691 32422 tablet_replica.cc:330] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: stopping tablet replica
I20260808 17:05:42.183900 32422 raft_consensus.cc:2229] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:05:42.183913 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 31691
I20260808 17:05:42.184036 32422 raft_consensus.cc:2258] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:42.191591 32422 ts_tablet_manager.cc:1905] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:42.193224 32422 ts_tablet_manager.cc:1918] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:42.193300 32422 log.cc:1196] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-1/wal/wals/6959f33bec39475f85f9927c6eb20ce3
I20260808 17:05:42.193545 32422 ts_tablet_manager.cc:1939] T 6959f33bec39475f85f9927c6eb20ce3 P 60bc252269a34b13a790ac05e06983b7: Deleting consensus metadata
I20260808 17:05:42.193943 31574 catalog_manager.cc:4841] TS 60bc252269a34b13a790ac05e06983b7 (127.30.194.2:36743): tablet 6959f33bec39475f85f9927c6eb20ce3 (table test-table [id=34f39ac5d38d48ffbeff2eccafbd3edc]) successfully deleted
W20260808 17:05:42.195207 31573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.194.1:35231 (error 108)
I20260808 17:05:42.195451 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 31930
I20260808 17:05:42.195699 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:42.195568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.1:35231 (local address 127.0.0.1:39284)
0808 17:05:42.195616 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:05:42.195666 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.1:35231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 17:05:42.197669 32420 tablet_replica.cc:330] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: stopping tablet replica
I20260808 17:05:42.197772 32420 raft_consensus.cc:2229] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:05:42.197872 32420 raft_consensus.cc:2258] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:05:42.198347 32420 ts_tablet_manager.cc:1905] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:05:42.200063 32420 ts_tablet_manager.cc:1918] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:05:42.200136 32420 log.cc:1196] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: Deleting WAL directory at /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208501350379-31496-0/minicluster-data/ts-2/wal/wals/6959f33bec39475f85f9927c6eb20ce3
I20260808 17:05:42.200379 32420 ts_tablet_manager.cc:1939] T 6959f33bec39475f85f9927c6eb20ce3 P e48e7c8d90c04f56949a5217ef950659: Deleting consensus metadata
W20260808 17:05:42.200628 31573 catalog_manager.cc:4584] TS c01de2a01e2c4d2ea7ba62295ae838c9 (127.30.194.1:35231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6959f33bec39475f85f9927c6eb20ce3: Network error: Client connection negotiation failed: client connection to 127.30.194.1:35231: connect: Connection refused (error 111)
I20260808 17:05:42.200816 31574 catalog_manager.cc:4841] TS e48e7c8d90c04f56949a5217ef950659 (127.30.194.3:38871): tablet 6959f33bec39475f85f9927c6eb20ce3 (table test-table [id=34f39ac5d38d48ffbeff2eccafbd3edc]) successfully deleted
I20260808 17:05:42.205749 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 32148
I20260808 17:05:42.216565 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 31548
2026-08-08T17:05:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (899 ms)
[----------] 6 tests from EncryptionPolicyTest (9129 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:42 dist-test-slave-2j7r krb5kdc[32427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:42 dist-test-slave-2j7r krb5kdc[32427](info): set up 2 sockets
Aug 08 17:05:42 dist-test-slave-2j7r krb5kdc[32427](info): commencing operation
krb5kdc: starting...
W20260808 17:05:43.383795 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:43 dist-test-slave-2j7r krb5kdc[32427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:43.491793 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33315
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:43.571848 32445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:43.572047 32445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:43.572072 32445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:43.573251 32445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:43.573295 32445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:43.573309 32445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:43.573323 32445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:43.573350 32445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:43.574821 32445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33315
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:43.575053 32445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:43.575280 32445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:43.579839 32453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:43.579859 32451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:43.579855 32450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:43.579953 32445 server_base.cc:1034] running on GCE node
I20260808 17:05:43.580217 32445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:43.580495 32445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:43.581642 32445 hybrid_clock.cc:648] HybridClock initialized: now 1786208743581626 us; error 32 us; skew 500 ppm
Aug 08 17:05:43 dist-test-slave-2j7r krb5kdc[32427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208743, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:43.584041 32445 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:43.584494 32445 webserver.cc:460] Webserver started at http://127.0.0.1:35259/ using document root <none> and password file <none>
I20260808 17:05:43.584702 32445 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:43.584753 32445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:43.584885 32445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:43.585923 32445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "c659aae23d444998a12bf2f3b0e73804"
format_stamp: "Formatted at 2026-08-08 17:05:43 on dist-test-slave-2j7r"
server_key: "5f0b2f7252f5a93969ec9104044a3cc6"
server_key_iv: "96d312a182479671959b2221f9548e18"
server_key_version: "encryptionkey@0"
I20260808 17:05:43.586297 32445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "c659aae23d444998a12bf2f3b0e73804"
format_stamp: "Formatted at 2026-08-08 17:05:43 on dist-test-slave-2j7r"
server_key: "5f0b2f7252f5a93969ec9104044a3cc6"
server_key_iv: "96d312a182479671959b2221f9548e18"
server_key_version: "encryptionkey@0"
I20260808 17:05:43.587563 32445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:05:43.588217 32466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:43.588379 32445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:43.588469 32445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "c659aae23d444998a12bf2f3b0e73804"
format_stamp: "Formatted at 2026-08-08 17:05:43 on dist-test-slave-2j7r"
server_key: "5f0b2f7252f5a93969ec9104044a3cc6"
server_key_iv: "96d312a182479671959b2221f9548e18"
server_key_version: "encryptionkey@0"
I20260808 17:05:43.588541 32445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:43.600723 32445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:43.602421 32445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:43.602552 32445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:43.606912 32445 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33315
I20260808 17:05:43.606973 32553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33315 every 8 connection(s)
I20260808 17:05:43.607260 32445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:43.608090 32554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:43.610316 32554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804: Bootstrap starting.
I20260808 17:05:43.610857 32554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:43.611112 32554 log.cc:824] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:43.611799 32554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804: No bootstrap required, opened a new log
I20260808 17:05:43.613379 32554 raft_consensus.cc:348] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659aae23d444998a12bf2f3b0e73804" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33315 } }
I20260808 17:05:43.613502 32554 raft_consensus.cc:374] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:43.613523 32554 raft_consensus.cc:729] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c659aae23d444998a12bf2f3b0e73804, State: Initialized, Role: FOLLOWER
I20260808 17:05:43.613627 32554 consensus_queue.cc:260] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659aae23d444998a12bf2f3b0e73804" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33315 } }
I20260808 17:05:43.613698 32554 raft_consensus.cc:388] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:43.613732 32554 raft_consensus.cc:482] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:43.613765 32554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:43.614212 32554 raft_consensus.cc:504] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659aae23d444998a12bf2f3b0e73804" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33315 } }
I20260808 17:05:43.614298 32554 leader_election.cc:302] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c659aae23d444998a12bf2f3b0e73804; no voters:
I20260808 17:05:43.614418 32554 leader_election.cc:288] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:43.614459 32559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:43.614635 32559 raft_consensus.cc:686] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [term 1 LEADER]: Becoming Leader. State: Replica: c659aae23d444998a12bf2f3b0e73804, State: Running, Role: LEADER
I20260808 17:05:43.614620 32554 sys_catalog.cc:564] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:43.614753 32559 consensus_queue.cc:237] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659aae23d444998a12bf2f3b0e73804" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33315 } }
I20260808 17:05:43.615108 32560 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c659aae23d444998a12bf2f3b0e73804" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659aae23d444998a12bf2f3b0e73804" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33315 } } }
I20260808 17:05:43.615185 32560 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:43.615206 32561 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c659aae23d444998a12bf2f3b0e73804. Latest consensus state: current_term: 1 leader_uuid: "c659aae23d444998a12bf2f3b0e73804" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659aae23d444998a12bf2f3b0e73804" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33315 } } }
I20260808 17:05:43.615253 32561 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:43.615410 32570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:43.615942 32570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:43.615970 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 32445
I20260808 17:05:43.616063 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:43.616240 31496 external_mini_cluster.cc:1428] Setting key 7521055878df831343c6bb2e2e6016ec
I20260808 17:05:43.617571 32570 catalog_manager.cc:1347] Generated new cluster ID: 9c84b2203d36459db776a065e8efd15f
I20260808 17:05:43.617624 32570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:05:43 dist-test-slave-2j7r krb5kdc[32427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:05:43.634075 32590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:43.617166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50715 (local address 127.0.0.1:33315)
0808 17:05:43.623935 (+ 6769us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:43.623944 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:43.623976 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:05:43.624082 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:43.624088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:43.624145 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:43.624271 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:43.625004 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:43.626345 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:43.627398 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:43.627967 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:43.628052 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:43.630669 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:43.630673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:43.630674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:43.630687 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:43.632480 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:43.632878 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:43.632881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:43.632882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:43.632984 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:43.633221 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:43.633222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:43.633223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:43.633514 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:43.633676 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:43.633762 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 17:05:43.633859 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6714,"thread_start_us":76,"threads_started":1}
I20260808 17:05:43.634924 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:43.638537 32570 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:43.638990 32570 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:43.643394 32590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:43.636334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60377 (local address 127.0.0.1:33315)
0808 17:05:43.636367 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:43.636369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:43.636482 (+ 113us) server_negotiation.cc:407] Connection header received
0808 17:05:43.636569 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:43.636570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:43.636578 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:43.636604 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:43.637313 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:43.638044 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:43.639045 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:43.639347 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:43.639383 (+ 36us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:43.640686 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:43.640688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:43.640688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:43.640694 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:43.642099 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:43.642505 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:43.642506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:43.642507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:43.642559 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:43.642801 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:43.642802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:43.642803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:43.642919 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:43.643024 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:43.643248 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 17:05:43.643309 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:05:43.648469 32570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c659aae23d444998a12bf2f3b0e73804: Generated new TSK 0
I20260808 17:05:43.648620 32570 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:43.651994 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 32445
W20260808 17:05:43.666814 32425 connection.cc:535] client connection to 127.0.0.1:33315 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33315): Connection reset by peer (error 104)
2026-08-08T17:05:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:43 dist-test-slave-2j7r krb5kdc[32606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:43 dist-test-slave-2j7r krb5kdc[32606](info): set up 2 sockets
Aug 08 17:05:43 dist-test-slave-2j7r krb5kdc[32606](info): commencing operation
krb5kdc: starting...
W20260808 17:05:44.796804 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:44 dist-test-slave-2j7r krb5kdc[32606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:44.905167 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36959
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:44.984377 32626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:44.984558 32626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:44.984579 32626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:44.985656 32626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:44.985689 32626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:44.985702 32626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:44.985713 32626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:44.985723 32626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:44.987114 32626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36959
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:44.987286 32626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:44.987443 32626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:44.991784 32635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:44.991894 32626 server_base.cc:1034] running on GCE node
W20260808 17:05:44.991802 32637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:44.991997 32639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:44.992199 32626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:44.992462 32626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:44.993603 32626 hybrid_clock.cc:648] HybridClock initialized: now 1786208744993586 us; error 27 us; skew 500 ppm
Aug 08 17:05:44 dist-test-slave-2j7r krb5kdc[32606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208744, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:44.995811 32626 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:44.996189 32626 webserver.cc:460] Webserver started at http://127.0.0.1:40709/ using document root <none> and password file <none>
I20260808 17:05:44.996394 32626 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:44.996441 32626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:44.996559 32626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:44.997546 32626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "b0d7aca9f675403bbc76edb63c590053"
format_stamp: "Formatted at 2026-08-08 17:05:44 on dist-test-slave-2j7r"
server_key: "fc35e7852e49905c6114286ebbe10210"
server_key_iv: "c151bd35e0004ad7e840f2258a32810f"
server_key_version: "encryptionkey@0"
I20260808 17:05:44.997849 32626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "b0d7aca9f675403bbc76edb63c590053"
format_stamp: "Formatted at 2026-08-08 17:05:44 on dist-test-slave-2j7r"
server_key: "fc35e7852e49905c6114286ebbe10210"
server_key_iv: "c151bd35e0004ad7e840f2258a32810f"
server_key_version: "encryptionkey@0"
I20260808 17:05:44.999028 32626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:05:44.999643 32654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:44.999785 32626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:44.999866 32626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "b0d7aca9f675403bbc76edb63c590053"
format_stamp: "Formatted at 2026-08-08 17:05:44 on dist-test-slave-2j7r"
server_key: "fc35e7852e49905c6114286ebbe10210"
server_key_iv: "c151bd35e0004ad7e840f2258a32810f"
server_key_version: "encryptionkey@0"
I20260808 17:05:44.999920 32626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:45.008816 32626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:45.010371 32626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:45.010494 32626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:45.014115 32626 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36959
I20260808 17:05:45.014156 32737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36959 every 8 connection(s)
I20260808 17:05:45.014443 32626 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:45.015221 32738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:45.017362 32738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053: Bootstrap starting.
I20260808 17:05:45.017894 32738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:45.018138 32738 log.cc:824] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:45.018743 32738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053: No bootstrap required, opened a new log
I20260808 17:05:45.019202 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 32626
I20260808 17:05:45.019302 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:45.019481 31496 external_mini_cluster.cc:1428] Setting key d61fcdaf0463ba764b3e024491cb283a
I20260808 17:05:45.020668 32738 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7aca9f675403bbc76edb63c590053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36959 } }
I20260808 17:05:45.020833 32738 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:45.020923 32738 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0d7aca9f675403bbc76edb63c590053, State: Initialized, Role: FOLLOWER
I20260808 17:05:45.021059 32738 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7aca9f675403bbc76edb63c590053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36959 } }
I20260808 17:05:45.021162 32738 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:45.021204 32738 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:45.021248 32738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:45.021915 32738 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7aca9f675403bbc76edb63c590053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36959 } }
I20260808 17:05:45.022020 32738 leader_election.cc:302] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0d7aca9f675403bbc76edb63c590053; no voters:
I20260808 17:05:45.022164 32738 leader_election.cc:288] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:45.022210 32744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:45.022368 32744 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [term 1 LEADER]: Becoming Leader. State: Replica: b0d7aca9f675403bbc76edb63c590053, State: Running, Role: LEADER
I20260808 17:05:45.022388 32738 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:45.022476 32744 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7aca9f675403bbc76edb63c590053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36959 } }
I20260808 17:05:45.022892 32744 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0d7aca9f675403bbc76edb63c590053. Latest consensus state: current_term: 1 leader_uuid: "b0d7aca9f675403bbc76edb63c590053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7aca9f675403bbc76edb63c590053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36959 } } }
I20260808 17:05:45.022964 32744 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:45.023176 32756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:45.023682 32745 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0d7aca9f675403bbc76edb63c590053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7aca9f675403bbc76edb63c590053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36959 } } }
I20260808 17:05:45.023773 32745 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:45.023823 32756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:45.025466 32756 catalog_manager.cc:1347] Generated new cluster ID: fb7539df96fc4e60b845ce989f7d598d
I20260808 17:05:45.025550 32756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:45.025975 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:45.019919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37459 (local address 127.0.0.1:36959)
0808 17:05:45.020094 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:45.020104 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:45.020137 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:05:45.020632 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:45.020641 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:45.020706 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:45.020843 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:45.021041 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:45.022644 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:45.023765 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:45.024337 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:45.024752 (+ 415us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:45.025242 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:45.025247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:45.025247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:45.025446 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:45.025483 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:45.025497 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:45.025723 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:45.025768 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 17:05:45.025797 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260808 17:05:45.026870 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:45.030644 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:45.028137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38543 (local address 127.0.0.1:36959)
0808 17:05:45.028207 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:45.028209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:45.028252 (+ 43us) server_negotiation.cc:407] Connection header received
0808 17:05:45.028306 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:45.028307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:45.028316 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:45.028359 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:45.028477 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:45.028908 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:45.029613 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:45.029832 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:45.029884 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:45.030423 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:45.030424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:45.030424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:45.030428 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:45.030441 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:45.030445 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:45.030490 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:45.030562 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 17:05:45.030572 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 17:05:45.037473 32756 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:45.037909 32756 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:45.039963 32756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0d7aca9f675403bbc76edb63c590053: Generated new TSK 0
I20260808 17:05:45.040124 32756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:45.048672 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 32626
W20260808 17:05:45.064447 32604 connection.cc:535] client connection to 127.0.0.1:36959 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36959): Connection reset by peer (error 104)
2026-08-08T17:05:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:45 dist-test-slave-2j7r krb5kdc[322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:45 dist-test-slave-2j7r krb5kdc[322](info): set up 2 sockets
Aug 08 17:05:45 dist-test-slave-2j7r krb5kdc[322](info): commencing operation
krb5kdc: starting...
W20260808 17:05:46.189425 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:46 dist-test-slave-2j7r krb5kdc[322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:46.297730 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35169
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:46.379818 345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:46.380002 345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:46.380029 345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:46.381158 345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:46.381213 345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:46.381232 345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:46.381249 345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:46.381273 345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:46.382712 345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35169
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:46.382944 345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:46.383134 345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:46.387609 356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:46.387601 354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:46.387602 359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:46.387683 345 server_base.cc:1034] running on GCE node
I20260808 17:05:46.387998 345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:46.388302 345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:46.389444 345 hybrid_clock.cc:648] HybridClock initialized: now 1786208746389428 us; error 32 us; skew 500 ppm
Aug 08 17:05:46 dist-test-slave-2j7r krb5kdc[322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208746, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:46.397280 345 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:46.397688 345 webserver.cc:460] Webserver started at http://127.0.0.1:45131/ using document root <none> and password file <none>
I20260808 17:05:46.397881 345 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:46.397923 345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:46.398043 345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:46.399001 345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "9eca190c645243c38b49eecda5ea46b2"
format_stamp: "Formatted at 2026-08-08 17:05:46 on dist-test-slave-2j7r"
server_key: "99ba195104306df82a00df5c8bf665dd"
server_key_iv: "7acf45bf0f8f2383c7bd9dafaefeceab"
server_key_version: "encryptionkey@0"
I20260808 17:05:46.399323 345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "9eca190c645243c38b49eecda5ea46b2"
format_stamp: "Formatted at 2026-08-08 17:05:46 on dist-test-slave-2j7r"
server_key: "99ba195104306df82a00df5c8bf665dd"
server_key_iv: "7acf45bf0f8f2383c7bd9dafaefeceab"
server_key_version: "encryptionkey@0"
I20260808 17:05:46.400496 345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:05:46.401117 377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:46.401285 345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:46.401371 345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "9eca190c645243c38b49eecda5ea46b2"
format_stamp: "Formatted at 2026-08-08 17:05:46 on dist-test-slave-2j7r"
server_key: "99ba195104306df82a00df5c8bf665dd"
server_key_iv: "7acf45bf0f8f2383c7bd9dafaefeceab"
server_key_version: "encryptionkey@0"
I20260808 17:05:46.401435 345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:46.404482 345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:46.406159 345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:46.406292 345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:46.409997 345 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35169
I20260808 17:05:46.410028 475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35169 every 8 connection(s)
I20260808 17:05:46.410320 345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:46.411206 476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:46.411582 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 345
I20260808 17:05:46.411695 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:46.411892 31496 external_mini_cluster.cc:1428] Setting key b390337b2e1a47d2002af576a1dc4ff7
I20260808 17:05:46.414014 476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2: Bootstrap starting.
I20260808 17:05:46.414145 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:46.412370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39211 (local address 127.0.0.1:35169)
0808 17:05:46.412530 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:46.412539 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:46.412562 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:05:46.412956 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:46.412962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:46.412985 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:46.413127 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:46.413699 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:46.413703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:46.413703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:46.413707 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:46.413743 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:46.413753 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:46.413804 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:46.413885 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 17:05:46.413914 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260808 17:05:46.414675 476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:46.414980 476 log.cc:824] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:46.415035 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:46.415637 476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2: No bootstrap required, opened a new log
I20260808 17:05:46.417263 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:46.416368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40215 (local address 127.0.0.1:35169)
0808 17:05:46.416410 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:46.416412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:46.416488 (+ 76us) server_negotiation.cc:407] Connection header received
0808 17:05:46.416524 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:46.416525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:46.416529 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:46.416592 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:46.417029 (+ 437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:46.417030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:46.417031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:46.417035 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:46.417049 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:46.417054 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:46.417124 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:46.417189 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 17:05:46.417202 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:05:46.417834 476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eca190c645243c38b49eecda5ea46b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35169 } }
I20260808 17:05:46.417968 476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:46.417989 476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eca190c645243c38b49eecda5ea46b2, State: Initialized, Role: FOLLOWER
I20260808 17:05:46.418077 476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eca190c645243c38b49eecda5ea46b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35169 } }
I20260808 17:05:46.418159 476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:46.418179 476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:46.418210 476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:46.418669 476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eca190c645243c38b49eecda5ea46b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35169 } }
I20260808 17:05:46.418754 476 leader_election.cc:302] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9eca190c645243c38b49eecda5ea46b2; no voters:
I20260808 17:05:46.418896 476 leader_election.cc:288] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:46.418941 490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:46.419111 490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [term 1 LEADER]: Becoming Leader. State: Replica: 9eca190c645243c38b49eecda5ea46b2, State: Running, Role: LEADER
I20260808 17:05:46.419113 476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:46.419263 490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eca190c645243c38b49eecda5ea46b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35169 } }
I20260808 17:05:46.419611 491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9eca190c645243c38b49eecda5ea46b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eca190c645243c38b49eecda5ea46b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35169 } } }
I20260808 17:05:46.419620 492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9eca190c645243c38b49eecda5ea46b2. Latest consensus state: current_term: 1 leader_uuid: "9eca190c645243c38b49eecda5ea46b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eca190c645243c38b49eecda5ea46b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35169 } } }
I20260808 17:05:46.419732 491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:46.419739 492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:46.420163 499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:46.420598 499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:46.421976 499 catalog_manager.cc:1347] Generated new cluster ID: 28c05ce183394ba18aa8e6edd314266b
I20260808 17:05:46.422019 499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:46.429522 499 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:46.429662 499 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:46.437928 499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9eca190c645243c38b49eecda5ea46b2: Generated new TSK 0
I20260808 17:05:46.438086 499 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:46.441387 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 345
2026-08-08T17:05:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:46 dist-test-slave-2j7r krb5kdc[524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:46 dist-test-slave-2j7r krb5kdc[524](info): set up 2 sockets
Aug 08 17:05:46 dist-test-slave-2j7r krb5kdc[524](info): commencing operation
krb5kdc: starting...
W20260808 17:05:47.612921 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:47 dist-test-slave-2j7r krb5kdc[524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:47.725222 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46449
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:47.804951 547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:47.805183 547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:47.805217 547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:47.806336 547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:47.806381 547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:47.806397 547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:47.806411 547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:47.806439 547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:47.807879 547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:47.808094 547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:47.808301 547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:47.812788 553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:47.812841 556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:47.812856 547 server_base.cc:1034] running on GCE node
W20260808 17:05:47.812795 554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:47.813236 547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:47.813524 547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:47.814666 547 hybrid_clock.cc:648] HybridClock initialized: now 1786208747814644 us; error 36 us; skew 500 ppm
Aug 08 17:05:47 dist-test-slave-2j7r krb5kdc[524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208747, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:47.816996 547 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:47.817435 547 webserver.cc:460] Webserver started at http://127.0.0.1:43007/ using document root <none> and password file <none>
I20260808 17:05:47.817619 547 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:47.817664 547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:47.817783 547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:47.818939 547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "f232771a78e74073b3554e4ced0891b2"
format_stamp: "Formatted at 2026-08-08 17:05:47 on dist-test-slave-2j7r"
server_key: "7cefe6bb38418bc316dfe53ad1eb38d1"
server_key_iv: "7631f5994db5681f0ce4d3f834618c00"
server_key_version: "encryptionkey@0"
I20260808 17:05:47.819314 547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "f232771a78e74073b3554e4ced0891b2"
format_stamp: "Formatted at 2026-08-08 17:05:47 on dist-test-slave-2j7r"
server_key: "7cefe6bb38418bc316dfe53ad1eb38d1"
server_key_iv: "7631f5994db5681f0ce4d3f834618c00"
server_key_version: "encryptionkey@0"
I20260808 17:05:47.820479 547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:05:47.821079 570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:47.821237 547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:47.821321 547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "f232771a78e74073b3554e4ced0891b2"
format_stamp: "Formatted at 2026-08-08 17:05:47 on dist-test-slave-2j7r"
server_key: "7cefe6bb38418bc316dfe53ad1eb38d1"
server_key_iv: "7631f5994db5681f0ce4d3f834618c00"
server_key_version: "encryptionkey@0"
I20260808 17:05:47.821391 547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:47.834386 547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:47.835987 547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:47.836102 547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:47.839818 547 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46449
I20260808 17:05:47.839848 658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46449 every 8 connection(s)
I20260808 17:05:47.840129 547 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:47.840879 662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:47.842903 662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2: Bootstrap starting.
I20260808 17:05:47.843441 662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:47.843689 662 log.cc:824] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:47.844312 662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2: No bootstrap required, opened a new log
I20260808 17:05:47.845849 662 raft_consensus.cc:348] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f232771a78e74073b3554e4ced0891b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46449 } }
I20260808 17:05:47.845988 662 raft_consensus.cc:374] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:47.846022 662 raft_consensus.cc:729] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f232771a78e74073b3554e4ced0891b2, State: Initialized, Role: FOLLOWER
I20260808 17:05:47.846148 662 consensus_queue.cc:260] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f232771a78e74073b3554e4ced0891b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46449 } }
I20260808 17:05:47.846215 662 raft_consensus.cc:388] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:47.846247 662 raft_consensus.cc:482] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:47.846292 662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:47.846757 662 raft_consensus.cc:504] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f232771a78e74073b3554e4ced0891b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46449 } }
I20260808 17:05:47.846844 662 leader_election.cc:302] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f232771a78e74073b3554e4ced0891b2; no voters:
I20260808 17:05:47.846988 662 leader_election.cc:288] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:47.847023 666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:47.847177 666 raft_consensus.cc:686] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [term 1 LEADER]: Becoming Leader. State: Replica: f232771a78e74073b3554e4ced0891b2, State: Running, Role: LEADER
I20260808 17:05:47.847213 662 sys_catalog.cc:564] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:47.847276 666 consensus_queue.cc:237] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f232771a78e74073b3554e4ced0891b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46449 } }
I20260808 17:05:47.847765 669 sys_catalog.cc:455] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f232771a78e74073b3554e4ced0891b2. Latest consensus state: current_term: 1 leader_uuid: "f232771a78e74073b3554e4ced0891b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f232771a78e74073b3554e4ced0891b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46449 } } }
I20260808 17:05:47.847852 669 sys_catalog.cc:458] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:47.848084 683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:47.848067 668 sys_catalog.cc:455] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f232771a78e74073b3554e4ced0891b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f232771a78e74073b3554e4ced0891b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46449 } } }
I20260808 17:05:47.848225 668 sys_catalog.cc:458] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:47.848580 683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:47.849283 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 547
I20260808 17:05:47.849397 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:47.849591 31496 external_mini_cluster.cc:1428] Setting key 56c5cc91126ba1e93cf5cf10fbc112fb
I20260808 17:05:47.850046 683 catalog_manager.cc:1347] Generated new cluster ID: 41a7784329e54c1e9ade35153b084286
I20260808 17:05:47.850095 683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:05:47 dist-test-slave-2j7r krb5kdc[524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:05:47.860358 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:47.850009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53073 (local address 127.0.0.1:46449)
0808 17:05:47.850310 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:47.850318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:47.850347 (+ 29us) server_negotiation.cc:407] Connection header received
0808 17:05:47.850453 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:47.850459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:47.850509 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:47.850637 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:47.851301 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:47.852714 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:47.853714 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:47.854320 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:47.854432 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:47.856895 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:47.856899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:47.856900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:47.856911 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:47.858852 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:47.859218 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:47.859221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:47.859221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:47.859327 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:47.859498 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:47.859499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:47.859499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:47.859783 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:47.859935 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:47.859998 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 17:05:47.860127 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":62,"threads_started":1}
I20260808 17:05:47.861280 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:47.864203 683 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:47.864651 683 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:47.869493 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:47.862634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49579 (local address 127.0.0.1:46449)
0808 17:05:47.862665 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:47.862667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:47.862771 (+ 104us) server_negotiation.cc:407] Connection header received
0808 17:05:47.862854 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:47.862855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:47.862864 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:47.862889 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:47.863529 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:47.864253 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:47.865299 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:47.865602 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:47.865639 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:47.866928 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:47.866930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:47.866930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:47.866936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:47.868291 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:47.868654 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:47.868656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:47.868656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:47.868705 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:47.868889 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:47.868890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:47.868890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:47.869001 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:47.869106 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:47.869322 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 17:05:47.869408 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:05:47.869997 683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f232771a78e74073b3554e4ced0891b2: Generated new TSK 0
I20260808 17:05:47.870134 683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:47.875802 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 547
2026-08-08T17:05:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:47 dist-test-slave-2j7r krb5kdc[711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:47 dist-test-slave-2j7r krb5kdc[711](info): set up 2 sockets
Aug 08 17:05:47 dist-test-slave-2j7r krb5kdc[711](info): commencing operation
krb5kdc: starting...
W20260808 17:05:49.048982 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:49 dist-test-slave-2j7r krb5kdc[711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:49.156777 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38627
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:49.236249 735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:49.236443 735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:49.236465 735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:49.237627 735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:49.237668 735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:49.237682 735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:49.237706 735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:49.237731 735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:49.239202 735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:49.239418 735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:49.239621 735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:49.244128 750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:49.244203 745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:49.244196 744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:49.244364 735 server_base.cc:1034] running on GCE node
I20260808 17:05:49.244587 735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:49.244890 735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:49.246022 735 hybrid_clock.cc:648] HybridClock initialized: now 1786208749246009 us; error 26 us; skew 500 ppm
Aug 08 17:05:49 dist-test-slave-2j7r krb5kdc[711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208749, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:49.248246 735 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:49.248647 735 webserver.cc:460] Webserver started at http://127.0.0.1:40137/ using document root <none> and password file <none>
I20260808 17:05:49.248857 735 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:49.248912 735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:49.249030 735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:49.250070 735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "96133b9f096f4e288e4cf5c8d216f280"
format_stamp: "Formatted at 2026-08-08 17:05:49 on dist-test-slave-2j7r"
server_key: "400cd84063ebad6e4532a0ed7ba7efaf"
server_key_iv: "a6b3f6b93782756fdca191693f2fbd7a"
server_key_version: "encryptionkey@0"
I20260808 17:05:49.250406 735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "96133b9f096f4e288e4cf5c8d216f280"
format_stamp: "Formatted at 2026-08-08 17:05:49 on dist-test-slave-2j7r"
server_key: "400cd84063ebad6e4532a0ed7ba7efaf"
server_key_iv: "a6b3f6b93782756fdca191693f2fbd7a"
server_key_version: "encryptionkey@0"
I20260808 17:05:49.251595 735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:05:49.252291 762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:49.252465 735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:49.252552 735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "96133b9f096f4e288e4cf5c8d216f280"
format_stamp: "Formatted at 2026-08-08 17:05:49 on dist-test-slave-2j7r"
server_key: "400cd84063ebad6e4532a0ed7ba7efaf"
server_key_iv: "a6b3f6b93782756fdca191693f2fbd7a"
server_key_version: "encryptionkey@0"
I20260808 17:05:49.252615 735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:49.267462 735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:49.269282 735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:49.269414 735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:49.272964 735 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38627
I20260808 17:05:49.273005 846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38627 every 8 connection(s)
I20260808 17:05:49.273336 735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:49.274140 847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:49.276330 847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280: Bootstrap starting.
I20260808 17:05:49.276834 847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:49.277060 847 log.cc:824] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:49.277716 847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280: No bootstrap required, opened a new log
I20260808 17:05:49.279245 847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96133b9f096f4e288e4cf5c8d216f280" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38627 } }
I20260808 17:05:49.279377 847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:49.279398 847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96133b9f096f4e288e4cf5c8d216f280, State: Initialized, Role: FOLLOWER
I20260808 17:05:49.279505 847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96133b9f096f4e288e4cf5c8d216f280" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38627 } }
I20260808 17:05:49.279578 847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:49.279605 847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:49.279637 847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:49.280089 847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96133b9f096f4e288e4cf5c8d216f280" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38627 } }
I20260808 17:05:49.280171 847 leader_election.cc:302] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96133b9f096f4e288e4cf5c8d216f280; no voters:
I20260808 17:05:49.280296 847 leader_election.cc:288] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:49.280341 853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:49.280524 847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:49.280527 853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [term 1 LEADER]: Becoming Leader. State: Replica: 96133b9f096f4e288e4cf5c8d216f280, State: Running, Role: LEADER
I20260808 17:05:49.280754 853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96133b9f096f4e288e4cf5c8d216f280" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38627 } }
I20260808 17:05:49.280761 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 735
I20260808 17:05:49.280875 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:49.281109 31496 external_mini_cluster.cc:1428] Setting key 6a26f26a49c187446f188ac7518dc585
I20260808 17:05:49.281231 859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96133b9f096f4e288e4cf5c8d216f280. Latest consensus state: current_term: 1 leader_uuid: "96133b9f096f4e288e4cf5c8d216f280" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96133b9f096f4e288e4cf5c8d216f280" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38627 } } }
I20260808 17:05:49.281311 859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:49.281508 868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:49.282048 868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:49.282316 858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96133b9f096f4e288e4cf5c8d216f280" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96133b9f096f4e288e4cf5c8d216f280" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38627 } } }
I20260808 17:05:49.282434 858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:49.285053 868 catalog_manager.cc:1347] Generated new cluster ID: e1f4f4e3ed464995834e0d0017f5fbb7
I20260808 17:05:49.285123 868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:49.287118 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:49.281795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58703 (local address 127.0.0.1:38627)
0808 17:05:49.282010 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:49.282020 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:49.282334 (+ 314us) server_negotiation.cc:407] Connection header received
0808 17:05:49.282742 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:49.282750 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:49.282814 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:49.282959 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:49.283204 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:49.284544 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:49.285452 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:49.285864 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:49.285977 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:49.286549 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:49.286553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:49.286553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:49.286715 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:49.286744 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:49.286755 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:49.286810 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:49.286888 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 17:05:49.286949 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":65,"threads_started":1}
I20260808 17:05:49.287949 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:49.291730 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:49.289239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53089 (local address 127.0.0.1:38627)
0808 17:05:49.289270 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:49.289272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:49.289441 (+ 169us) server_negotiation.cc:407] Connection header received
0808 17:05:49.289493 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:49.289494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:49.289504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:49.289544 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:49.289657 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:49.290111 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:49.290756 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:49.290970 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:49.291023 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:49.291529 (+ 506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:49.291530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:49.291531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:49.291535 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:49.291548 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:49.291552 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:49.291592 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:49.291639 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 17:05:49.291662 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 17:05:49.301710 868 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:49.302171 868 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:49.307261 868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96133b9f096f4e288e4cf5c8d216f280: Generated new TSK 0
I20260808 17:05:49.307427 868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:49.316481 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 735
2026-08-08T17:05:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:49 dist-test-slave-2j7r krb5kdc[903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:49 dist-test-slave-2j7r krb5kdc[903](info): set up 2 sockets
Aug 08 17:05:49 dist-test-slave-2j7r krb5kdc[903](info): commencing operation
krb5kdc: starting...
W20260808 17:05:50.480382 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:50 dist-test-slave-2j7r krb5kdc[903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:50.585306 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:50.663656 924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:50.663848 924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:50.663887 924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:50.664965 924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:50.665004 924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:50.665023 924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:50.665040 924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:50.665055 924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:50.666515 924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:50.666749 924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:50.666949 924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:50.671447 936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:50.671458 932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:50.671510 924 server_base.cc:1034] running on GCE node
W20260808 17:05:50.671458 933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:50.671854 924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:50.672079 924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:50.673192 924 hybrid_clock.cc:648] HybridClock initialized: now 1786208750673182 us; error 27 us; skew 500 ppm
Aug 08 17:05:50 dist-test-slave-2j7r krb5kdc[903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208750, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:50.675400 924 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:50.675804 924 webserver.cc:460] Webserver started at http://127.0.0.1:36111/ using document root <none> and password file <none>
I20260808 17:05:50.676004 924 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:50.676046 924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:50.676162 924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:50.677140 924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "08e2188fd9914e64b786d15cf6dc7776"
format_stamp: "Formatted at 2026-08-08 17:05:50 on dist-test-slave-2j7r"
server_key: "2bb2dce30536a9bd8407a63c19fb3fd8"
server_key_iv: "6a0ab8bd54ee240b68b4f1d881df2474"
server_key_version: "encryptionkey@0"
I20260808 17:05:50.677460 924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "08e2188fd9914e64b786d15cf6dc7776"
format_stamp: "Formatted at 2026-08-08 17:05:50 on dist-test-slave-2j7r"
server_key: "2bb2dce30536a9bd8407a63c19fb3fd8"
server_key_iv: "6a0ab8bd54ee240b68b4f1d881df2474"
server_key_version: "encryptionkey@0"
I20260808 17:05:50.678633 924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:05:50.679293 947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:50.679430 924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:50.679515 924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "08e2188fd9914e64b786d15cf6dc7776"
format_stamp: "Formatted at 2026-08-08 17:05:50 on dist-test-slave-2j7r"
server_key: "2bb2dce30536a9bd8407a63c19fb3fd8"
server_key_iv: "6a0ab8bd54ee240b68b4f1d881df2474"
server_key_version: "encryptionkey@0"
I20260808 17:05:50.679579 924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:50.682703 924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:50.684355 924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:50.684476 924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:50.688115 924 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41109
I20260808 17:05:50.688143 1037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41109 every 8 connection(s)
I20260808 17:05:50.688414 924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:50.689050 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 924
I20260808 17:05:50.689163 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:50.689222 1040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:50.689379 31496 external_mini_cluster.cc:1428] Setting key 0198f6c92f1c8397ae2d8c1633d115f2
I20260808 17:05:50.691653 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:50.689959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40861 (local address 127.0.0.1:41109)
0808 17:05:50.690113 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:50.690120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:50.690142 (+ 22us) server_negotiation.cc:407] Connection header received
0808 17:05:50.690581 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:50.690587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:50.690610 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:50.690711 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:50.691276 (+ 565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:50.691279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:50.691279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:50.691282 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:50.691315 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:50.691325 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:50.691375 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:50.691450 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 17:05:50.691482 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260808 17:05:50.692241 1040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776: Bootstrap starting.
I20260808 17:05:50.692528 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:50.692873 1040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:50.693179 1040 log.cc:824] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:50.693866 1040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776: No bootstrap required, opened a new log
I20260808 17:05:50.694779 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:50.693848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58969 (local address 127.0.0.1:41109)
0808 17:05:50.693878 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:50.693880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:50.693998 (+ 118us) server_negotiation.cc:407] Connection header received
0808 17:05:50.694061 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:50.694062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:50.694066 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:50.694110 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:50.694589 (+ 479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:50.694590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:50.694590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:50.694594 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:50.694605 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:50.694609 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:50.694653 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:50.694701 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 17:05:50.694712 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 17:05:50.695667 1040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e2188fd9914e64b786d15cf6dc7776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260808 17:05:50.695788 1040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:50.695808 1040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e2188fd9914e64b786d15cf6dc7776, State: Initialized, Role: FOLLOWER
I20260808 17:05:50.695895 1040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e2188fd9914e64b786d15cf6dc7776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260808 17:05:50.695974 1040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:50.696000 1040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:50.696033 1040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:50.696476 1040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e2188fd9914e64b786d15cf6dc7776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260808 17:05:50.696560 1040 leader_election.cc:302] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08e2188fd9914e64b786d15cf6dc7776; no voters:
I20260808 17:05:50.696672 1040 leader_election.cc:288] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:50.696708 1055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:50.696832 1055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [term 1 LEADER]: Becoming Leader. State: Replica: 08e2188fd9914e64b786d15cf6dc7776, State: Running, Role: LEADER
I20260808 17:05:50.696888 1040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:50.696942 1055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e2188fd9914e64b786d15cf6dc7776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260808 17:05:50.697335 1056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08e2188fd9914e64b786d15cf6dc7776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e2188fd9914e64b786d15cf6dc7776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } } }
I20260808 17:05:50.697425 1056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:50.697631 1057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08e2188fd9914e64b786d15cf6dc7776. Latest consensus state: current_term: 1 leader_uuid: "08e2188fd9914e64b786d15cf6dc7776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e2188fd9914e64b786d15cf6dc7776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } } }
I20260808 17:05:50.697660 1065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:50.697736 1057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:50.698194 1065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:50.699703 1065 catalog_manager.cc:1347] Generated new cluster ID: 5a9ed8c7dd214e6b93a0fbef6c5cd653
I20260808 17:05:50.699747 1065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:50.714804 1065 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:50.714957 1065 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:50.718472 1065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08e2188fd9914e64b786d15cf6dc7776: Generated new TSK 0
I20260808 17:05:50.718623 1065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:50.720729 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 924
2026-08-08T17:05:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:50 dist-test-slave-2j7r krb5kdc[1084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:50 dist-test-slave-2j7r krb5kdc[1084](info): set up 2 sockets
Aug 08 17:05:50 dist-test-slave-2j7r krb5kdc[1084](info): commencing operation
krb5kdc: starting...
W20260808 17:05:51.869266 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:51 dist-test-slave-2j7r krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:51.977195 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33401
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:52.056038 1111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:52.056224 1111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:52.056244 1111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:52.057374 1111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:52.057413 1111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:52.057427 1111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:52.057443 1111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:52.057459 1111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:52.058943 1111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:52.059155 1111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:52.059360 1111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:52.063839 1124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:52.063853 1123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:52.063853 1127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:52.063977 1111 server_base.cc:1034] running on GCE node
I20260808 17:05:52.064301 1111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:52.064561 1111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:52.065690 1111 hybrid_clock.cc:648] HybridClock initialized: now 1786208752065679 us; error 28 us; skew 500 ppm
Aug 08 17:05:52 dist-test-slave-2j7r krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:52.072597 1111 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:52.073011 1111 webserver.cc:460] Webserver started at http://127.0.0.1:43433/ using document root <none> and password file <none>
I20260808 17:05:52.073215 1111 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:52.073259 1111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:52.073346 1111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:52.074278 1111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "43ac9e1c969b402b8ce0f610845fa905"
format_stamp: "Formatted at 2026-08-08 17:05:52 on dist-test-slave-2j7r"
server_key: "8d176bbabb3de9e3f8a22ae336b7f6a5"
server_key_iv: "7dcb71fe973f5fee231f2172859fe7bc"
server_key_version: "encryptionkey@0"
I20260808 17:05:52.074566 1111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "43ac9e1c969b402b8ce0f610845fa905"
format_stamp: "Formatted at 2026-08-08 17:05:52 on dist-test-slave-2j7r"
server_key: "8d176bbabb3de9e3f8a22ae336b7f6a5"
server_key_iv: "7dcb71fe973f5fee231f2172859fe7bc"
server_key_version: "encryptionkey@0"
I20260808 17:05:52.075734 1111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:05:52.076349 1136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:52.076512 1111 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:52.076601 1111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "43ac9e1c969b402b8ce0f610845fa905"
format_stamp: "Formatted at 2026-08-08 17:05:52 on dist-test-slave-2j7r"
server_key: "8d176bbabb3de9e3f8a22ae336b7f6a5"
server_key_iv: "7dcb71fe973f5fee231f2172859fe7bc"
server_key_version: "encryptionkey@0"
I20260808 17:05:52.076661 1111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:52.092442 1111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:52.094502 1111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:52.094616 1111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:52.098467 1111 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33401
I20260808 17:05:52.098506 1220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33401 every 8 connection(s)
I20260808 17:05:52.098815 1111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:52.099604 1221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:52.101073 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 1111
I20260808 17:05:52.101228 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:52.101418 31496 external_mini_cluster.cc:1428] Setting key a73d41909117c3c9d28800c91c9ddc8f
I20260808 17:05:52.101965 1221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905: Bootstrap starting.
I20260808 17:05:52.102535 1221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:52.102795 1221 log.cc:824] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:52.103542 1221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905: No bootstrap required, opened a new log
I20260808 17:05:52.105521 1221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac9e1c969b402b8ce0f610845fa905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } }
I20260808 17:05:52.105652 1221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:52.105674 1221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ac9e1c969b402b8ce0f610845fa905, State: Initialized, Role: FOLLOWER
I20260808 17:05:52.105778 1221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac9e1c969b402b8ce0f610845fa905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } }
I20260808 17:05:52.105836 1221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:52.105856 1221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:52.105901 1221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:52.106526 1221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac9e1c969b402b8ce0f610845fa905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } }
I20260808 17:05:52.106633 1221 leader_election.cc:302] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43ac9e1c969b402b8ce0f610845fa905; no voters:
I20260808 17:05:52.106798 1221 leader_election.cc:288] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:52.106830 1236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:52.106985 1236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [term 1 LEADER]: Becoming Leader. State: Replica: 43ac9e1c969b402b8ce0f610845fa905, State: Running, Role: LEADER
I20260808 17:05:52.107048 1221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:52.107134 1236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac9e1c969b402b8ce0f610845fa905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } }
I20260808 17:05:52.107471 1236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43ac9e1c969b402b8ce0f610845fa905. Latest consensus state: current_term: 1 leader_uuid: "43ac9e1c969b402b8ce0f610845fa905" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac9e1c969b402b8ce0f610845fa905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } } }
I20260808 17:05:52.107674 1236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:52.107614 1240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43ac9e1c969b402b8ce0f610845fa905" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac9e1c969b402b8ce0f610845fa905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } } }
I20260808 17:05:52.108028 1240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905 [sys.catalog]: This master's current role is: LEADER
Aug 08 17:05:52 dist-test-slave-2j7r krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
W20260808 17:05:52.108848 1268 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:05:52.108899 1268 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:05:52.108942 1255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:52.109127 1255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:52.110558 1255 catalog_manager.cc:1347] Generated new cluster ID: f18d18fb224a426fae5d47050df88a95
I20260808 17:05:52.110603 1255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:52.112484 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:52.102007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:53207 (local address 127.0.0.1:33401)
0808 17:05:52.102168 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:52.102176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:52.102205 (+ 29us) server_negotiation.cc:407] Connection header received
0808 17:05:52.102618 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:52.102626 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:52.102701 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:52.102821 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:52.103479 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:52.104382 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:52.105471 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:52.105986 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:52.106073 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:52.108689 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:52.108693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:52.108694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:52.108704 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:52.110641 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:52.111103 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:52.111107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:52.111107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:52.111205 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:52.111452 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:52.111454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:52.111454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:52.111741 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:52.111866 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:52.112180 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 17:05:52.112282 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260808 17:05:52.113647 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:52.115623 1255 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:52.116307 1255 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:52.121124 1255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43ac9e1c969b402b8ce0f610845fa905: Generated new TSK 0
I20260808 17:05:52.121277 1255 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:52.121886 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:52.115072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:51007 (local address 127.0.0.1:33401)
0808 17:05:52.115113 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:52.115114 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:52.115417 (+ 303us) server_negotiation.cc:407] Connection header received
0808 17:05:52.115471 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:52.115471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:52.115480 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:52.115524 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:52.116158 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:52.116832 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:52.117797 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:52.118080 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:52.118111 (+ 31us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:52.119354 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:52.119355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:52.119355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:52.119361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:52.120697 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:52.121159 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:52.121160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:52.121161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:52.121214 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:52.121444 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:52.121445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:52.121445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:52.121540 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:52.121622 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:52.121754 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 17:05:52.121810 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:05:52.123406 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 1111
W20260808 17:05:52.145954 1077 connection.cc:535] client connection to 127.0.0.1:33401 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33401): Connection reset by peer (error 104)
2026-08-08T17:05:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:52 dist-test-slave-2j7r krb5kdc[1283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:52 dist-test-slave-2j7r krb5kdc[1283](info): set up 2 sockets
Aug 08 17:05:52 dist-test-slave-2j7r krb5kdc[1283](info): commencing operation
krb5kdc: starting...
W20260808 17:05:53.283429 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:53 dist-test-slave-2j7r krb5kdc[1283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:53.399593 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:53.478139 1305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:53.478318 1305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:53.478339 1305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:53.479390 1305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:53.479423 1305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:53.479436 1305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:53.479447 1305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:53.479457 1305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:53.480825 1305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:53.481016 1305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:53.481215 1305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:53.485500 1316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:53.485507 1318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:53.485514 1320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:53.485742 1305 server_base.cc:1034] running on GCE node
I20260808 17:05:53.485970 1305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:53.486222 1305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:53.487345 1305 hybrid_clock.cc:648] HybridClock initialized: now 1786208753487332 us; error 30 us; skew 500 ppm
Aug 08 17:05:53 dist-test-slave-2j7r krb5kdc[1283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208753, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:53.489648 1305 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:53.490063 1305 webserver.cc:460] Webserver started at http://127.0.0.1:43575/ using document root <none> and password file <none>
I20260808 17:05:53.490263 1305 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:53.490309 1305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:53.490430 1305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:53.491389 1305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "56ecfb30d2334a0ca3135e909361410f"
format_stamp: "Formatted at 2026-08-08 17:05:53 on dist-test-slave-2j7r"
server_key: "894e7c1430ebfa55ef922c1e7dba66ca"
server_key_iv: "fed4091a06c2052cef6f0d0747c6cd28"
server_key_version: "encryptionkey@0"
I20260808 17:05:53.491701 1305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "56ecfb30d2334a0ca3135e909361410f"
format_stamp: "Formatted at 2026-08-08 17:05:53 on dist-test-slave-2j7r"
server_key: "894e7c1430ebfa55ef922c1e7dba66ca"
server_key_iv: "fed4091a06c2052cef6f0d0747c6cd28"
server_key_version: "encryptionkey@0"
I20260808 17:05:53.492926 1305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:05:53.493618 1331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:53.493758 1305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:53.493839 1305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "56ecfb30d2334a0ca3135e909361410f"
format_stamp: "Formatted at 2026-08-08 17:05:53 on dist-test-slave-2j7r"
server_key: "894e7c1430ebfa55ef922c1e7dba66ca"
server_key_iv: "fed4091a06c2052cef6f0d0747c6cd28"
server_key_version: "encryptionkey@0"
I20260808 17:05:53.493907 1305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:53.501365 1305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:53.502866 1305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:53.502995 1305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:53.506590 1305 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40671
I20260808 17:05:53.506623 1435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40671 every 8 connection(s)
I20260808 17:05:53.506917 1305 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:53.507668 1436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:53.509830 1436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f: Bootstrap starting.
I20260808 17:05:53.510358 1436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:53.510603 1436 log.cc:824] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:53.511222 1436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f: No bootstrap required, opened a new log
I20260808 17:05:53.512717 1436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ecfb30d2334a0ca3135e909361410f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } }
I20260808 17:05:53.512843 1436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:53.512866 1436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56ecfb30d2334a0ca3135e909361410f, State: Initialized, Role: FOLLOWER
I20260808 17:05:53.512984 1436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ecfb30d2334a0ca3135e909361410f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } }
I20260808 17:05:53.513044 1436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:53.513064 1436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:53.513123 1436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:53.513622 1436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ecfb30d2334a0ca3135e909361410f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } }
I20260808 17:05:53.513702 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 1305
I20260808 17:05:53.513715 1436 leader_election.cc:302] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56ecfb30d2334a0ca3135e909361410f; no voters:
I20260808 17:05:53.513823 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:53.513859 1436 leader_election.cc:288] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:53.513895 1443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:53.514021 31496 external_mini_cluster.cc:1428] Setting key a364563e1ac1d07fc5b8063457904ce0
I20260808 17:05:53.514041 1443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [term 1 LEADER]: Becoming Leader. State: Replica: 56ecfb30d2334a0ca3135e909361410f, State: Running, Role: LEADER
I20260808 17:05:53.514115 1443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ecfb30d2334a0ca3135e909361410f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } }
I20260808 17:05:53.514163 1436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:53.514454 1445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56ecfb30d2334a0ca3135e909361410f. Latest consensus state: current_term: 1 leader_uuid: "56ecfb30d2334a0ca3135e909361410f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ecfb30d2334a0ca3135e909361410f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } } }
I20260808 17:05:53.514532 1445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:53.514544 1444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56ecfb30d2334a0ca3135e909361410f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ecfb30d2334a0ca3135e909361410f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } } }
I20260808 17:05:53.514614 1444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:53.514847 1453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:53.515676 1453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:53.518585 1453 catalog_manager.cc:1347] Generated new cluster ID: e08b807f71d54d57b6236244ebab0d1f
I20260808 17:05:53.518638 1453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:53.522514 1452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:53.514548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:40095 (local address 127.0.0.1:40671)
0808 17:05:53.514738 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:53.514747 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:53.514949 (+ 202us) server_negotiation.cc:407] Connection header received
0808 17:05:53.515086 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:53.515094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:53.515154 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:53.516033 (+ 879us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:53.516099 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:53.516937 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:53.517905 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:53.518478 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:53.521235 (+ 2757us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:53.521878 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:53.521883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:53.521883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:53.522078 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:53.522116 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:53.522130 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:53.522226 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:53.522295 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 17:05:53.522324 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260808 17:05:53.523545 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:53.528136 1452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:53.524921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:33893 (local address 127.0.0.1:40671)
0808 17:05:53.524957 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:53.524958 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:53.525052 (+ 94us) server_negotiation.cc:407] Connection header received
0808 17:05:53.525108 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:53.525109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:53.525119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:53.525183 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:53.525307 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:53.526049 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:53.526974 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:53.527291 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:53.527349 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:53.527915 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:53.527916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:53.527917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:53.527921 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:53.527934 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:53.527939 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:53.527991 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:53.528054 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 17:05:53.528065 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:05:53.534777 1453 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:53.535228 1453 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:53.546185 1453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56ecfb30d2334a0ca3135e909361410f: Generated new TSK 0
I20260808 17:05:53.546344 1453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:53.555524 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 1305
W20260808 17:05:53.568591 1279 connection.cc:535] client connection to 127.0.0.1:40671 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40671): Connection reset by peer (error 104)
2026-08-08T17:05:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:53 dist-test-slave-2j7r krb5kdc[1485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:53 dist-test-slave-2j7r krb5kdc[1485](info): set up 2 sockets
Aug 08 17:05:53 dist-test-slave-2j7r krb5kdc[1485](info): commencing operation
krb5kdc: starting...
W20260808 17:05:54.702255 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:54 dist-test-slave-2j7r krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:54.809600 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42711
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:54.889146 1503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:54.889334 1503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:54.889364 1503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:54.890451 1503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:54.890491 1503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:54.890511 1503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:54.890525 1503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:54.890542 1503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:54.891974 1503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:54.892207 1503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:54.892393 1503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:54.896838 1517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:54.896855 1513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:54.896873 1512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:54.896935 1503 server_base.cc:1034] running on GCE node
I20260808 17:05:54.897294 1503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:54.897554 1503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:54.898693 1503 hybrid_clock.cc:648] HybridClock initialized: now 1786208754898672 us; error 33 us; skew 500 ppm
Aug 08 17:05:54 dist-test-slave-2j7r krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208754, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:54.900969 1503 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:54.901408 1503 webserver.cc:460] Webserver started at http://127.0.0.1:42061/ using document root <none> and password file <none>
I20260808 17:05:54.901612 1503 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:54.901659 1503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:54.901779 1503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:54.902716 1503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "7f7acfdd89a345e197d750e38d10ea13"
format_stamp: "Formatted at 2026-08-08 17:05:54 on dist-test-slave-2j7r"
server_key: "c87f9750fcaf2c7b0709d4a838edac66"
server_key_iv: "803d504400a0ec6143d6ba7ac651d052"
server_key_version: "encryptionkey@0"
I20260808 17:05:54.903060 1503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "7f7acfdd89a345e197d750e38d10ea13"
format_stamp: "Formatted at 2026-08-08 17:05:54 on dist-test-slave-2j7r"
server_key: "c87f9750fcaf2c7b0709d4a838edac66"
server_key_iv: "803d504400a0ec6143d6ba7ac651d052"
server_key_version: "encryptionkey@0"
I20260808 17:05:54.904243 1503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:05:54.904887 1527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:54.905042 1503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:54.905139 1503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "7f7acfdd89a345e197d750e38d10ea13"
format_stamp: "Formatted at 2026-08-08 17:05:54 on dist-test-slave-2j7r"
server_key: "c87f9750fcaf2c7b0709d4a838edac66"
server_key_iv: "803d504400a0ec6143d6ba7ac651d052"
server_key_version: "encryptionkey@0"
I20260808 17:05:54.905206 1503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:54.908274 1503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:54.910079 1503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:54.910204 1503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:54.913939 1503 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42711
I20260808 17:05:54.913969 1618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42711 every 8 connection(s)
I20260808 17:05:54.914248 1503 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:54.915012 1620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:54.916975 1620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13: Bootstrap starting.
I20260808 17:05:54.917500 1620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:54.917728 1620 log.cc:824] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:54.918291 1620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13: No bootstrap required, opened a new log
I20260808 17:05:54.919759 1620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7acfdd89a345e197d750e38d10ea13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } }
I20260808 17:05:54.919876 1620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:54.919896 1620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f7acfdd89a345e197d750e38d10ea13, State: Initialized, Role: FOLLOWER
I20260808 17:05:54.919996 1620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7acfdd89a345e197d750e38d10ea13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } }
I20260808 17:05:54.920068 1620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:54.920092 1620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:54.920122 1620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:54.920542 1620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7acfdd89a345e197d750e38d10ea13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } }
I20260808 17:05:54.920621 1620 leader_election.cc:302] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f7acfdd89a345e197d750e38d10ea13; no voters:
I20260808 17:05:54.920740 1620 leader_election.cc:288] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:54.920782 1626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:54.920938 1620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:54.920959 1626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [term 1 LEADER]: Becoming Leader. State: Replica: 7f7acfdd89a345e197d750e38d10ea13, State: Running, Role: LEADER
I20260808 17:05:54.921038 1626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7acfdd89a345e197d750e38d10ea13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } }
I20260808 17:05:54.921406 1627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f7acfdd89a345e197d750e38d10ea13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7acfdd89a345e197d750e38d10ea13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } } }
I20260808 17:05:54.921428 1628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f7acfdd89a345e197d750e38d10ea13. Latest consensus state: current_term: 1 leader_uuid: "7f7acfdd89a345e197d750e38d10ea13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7acfdd89a345e197d750e38d10ea13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } } }
I20260808 17:05:54.921558 1628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:54.921545 1627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:54.921893 1632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:54.922395 1632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:54.923444 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 1503
I20260808 17:05:54.923550 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:54.923732 31496 external_mini_cluster.cc:1428] Setting key e255bd7ad68506512d23fe8212c7864c
I20260808 17:05:54.924134 1632 catalog_manager.cc:1347] Generated new cluster ID: 444147956b64406692db52aab6a86712
I20260808 17:05:54.924234 1632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:54.925617 1654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:54.924114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:48955 (local address 127.0.0.1:42711)
0808 17:05:54.924285 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:54.924293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:54.924325 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:05:54.924451 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:54.924459 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:54.924495 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:54.924653 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:54.925195 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:54.925199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:54.925199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:54.925203 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:54.925244 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:54.925257 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:54.925308 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:54.925405 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 17:05:54.925443 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260808 17:05:54.926625 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:54.928989 1654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:54.928106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:59839 (local address 127.0.0.1:42711)
0808 17:05:54.928142 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:54.928144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:54.928152 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:54.928171 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:54.928171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:54.928175 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:54.928236 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:54.928687 (+ 451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:54.928688 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:54.928688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:54.928692 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:54.928707 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:54.928712 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:54.928771 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:54.928820 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 17:05:54.928832 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:05:54.932830 1632 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:54.932973 1632 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:54.938895 1632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f7acfdd89a345e197d750e38d10ea13: Generated new TSK 0
I20260808 17:05:54.939040 1632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:54.944793 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 1503
2026-08-08T17:05:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:55 dist-test-slave-2j7r krb5kdc[1666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:55 dist-test-slave-2j7r krb5kdc[1666](info): set up 2 sockets
Aug 08 17:05:55 dist-test-slave-2j7r krb5kdc[1666](info): commencing operation
krb5kdc: starting...
W20260808 17:05:56.089991 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:56 dist-test-slave-2j7r krb5kdc[1666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:56.197340 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:56.276127 1686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:56.276312 1686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:56.276343 1686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:56.277482 1686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:56.277522 1686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:56.277542 1686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:56.277558 1686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:56.277580 1686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:56.279016 1686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33759
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:56.279249 1686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:56.279440 1686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:05:56.283938 1686 server_base.cc:1034] running on GCE node
W20260808 17:05:56.283890 1703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:56.283906 1697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:56.283960 1698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:56.284415 1686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:56.284684 1686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:56.285820 1686 hybrid_clock.cc:648] HybridClock initialized: now 1786208756285803 us; error 26 us; skew 500 ppm
Aug 08 17:05:56 dist-test-slave-2j7r krb5kdc[1666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208756, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:56.288024 1686 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:56.288460 1686 webserver.cc:460] Webserver started at http://127.0.0.1:44425/ using document root <none> and password file <none>
I20260808 17:05:56.288655 1686 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:56.288697 1686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:56.288810 1686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:56.289790 1686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "e93fd66675854efd95f7840f1c705e67"
format_stamp: "Formatted at 2026-08-08 17:05:56 on dist-test-slave-2j7r"
server_key: "c88f599459faa1687c49439329b64871"
server_key_iv: "e01df57c8dc99226613e6069d9b30ad4"
server_key_version: "encryptionkey@0"
I20260808 17:05:56.290098 1686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "e93fd66675854efd95f7840f1c705e67"
format_stamp: "Formatted at 2026-08-08 17:05:56 on dist-test-slave-2j7r"
server_key: "c88f599459faa1687c49439329b64871"
server_key_iv: "e01df57c8dc99226613e6069d9b30ad4"
server_key_version: "encryptionkey@0"
I20260808 17:05:56.291311 1686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:05:56.291982 1716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:56.292153 1686 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:56.292243 1686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "e93fd66675854efd95f7840f1c705e67"
format_stamp: "Formatted at 2026-08-08 17:05:56 on dist-test-slave-2j7r"
server_key: "c88f599459faa1687c49439329b64871"
server_key_iv: "e01df57c8dc99226613e6069d9b30ad4"
server_key_version: "encryptionkey@0"
I20260808 17:05:56.292299 1686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:56.299751 1686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:56.301419 1686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:56.301517 1686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:56.304951 1686 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33759
I20260808 17:05:56.304982 1807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33759 every 8 connection(s)
I20260808 17:05:56.305322 1686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:56.306140 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:56.308184 1808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67: Bootstrap starting.
I20260808 17:05:56.308709 1808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:56.308961 1808 log.cc:824] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:56.309608 1808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67: No bootstrap required, opened a new log
I20260808 17:05:56.311101 1808 raft_consensus.cc:348] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93fd66675854efd95f7840f1c705e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } }
I20260808 17:05:56.311187 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 1686
I20260808 17:05:56.311229 1808 raft_consensus.cc:374] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:56.311262 1808 raft_consensus.cc:729] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e93fd66675854efd95f7840f1c705e67, State: Initialized, Role: FOLLOWER
I20260808 17:05:56.311285 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:56.311439 31496 external_mini_cluster.cc:1428] Setting key e2a573be73d08b42566369b9039c625b
I20260808 17:05:56.311396 1808 consensus_queue.cc:260] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93fd66675854efd95f7840f1c705e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } }
I20260808 17:05:56.311472 1808 raft_consensus.cc:388] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:56.311502 1808 raft_consensus.cc:482] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:56.311550 1808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:56.312386 1808 raft_consensus.cc:504] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93fd66675854efd95f7840f1c705e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } }
I20260808 17:05:56.312491 1808 leader_election.cc:302] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e93fd66675854efd95f7840f1c705e67; no voters:
I20260808 17:05:56.312645 1808 leader_election.cc:288] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:56.312680 1819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:56.312776 1819 raft_consensus.cc:686] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [term 1 LEADER]: Becoming Leader. State: Replica: e93fd66675854efd95f7840f1c705e67, State: Running, Role: LEADER
I20260808 17:05:56.312901 1808 sys_catalog.cc:564] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:56.312885 1819 consensus_queue.cc:237] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93fd66675854efd95f7840f1c705e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } }
I20260808 17:05:56.313256 1820 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e93fd66675854efd95f7840f1c705e67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93fd66675854efd95f7840f1c705e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } } }
I20260808 17:05:56.313257 1819 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e93fd66675854efd95f7840f1c705e67. Latest consensus state: current_term: 1 leader_uuid: "e93fd66675854efd95f7840f1c705e67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93fd66675854efd95f7840f1c705e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } } }
I20260808 17:05:56.313359 1820 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:56.313364 1819 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:56.313613 1829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:56.314169 1829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:56.316475 1829 catalog_manager.cc:1347] Generated new cluster ID: 5db127ca9d2a4befa970d471dd288e96
I20260808 17:05:56.316524 1829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:05:56 dist-test-slave-2j7r krb5kdc[1666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:05:56.322487 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:56.311866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:38385 (local address 127.0.0.1:33759)
0808 17:05:56.312051 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:56.312060 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:56.312086 (+ 26us) server_negotiation.cc:407] Connection header received
0808 17:05:56.312173 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:56.312177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:56.312217 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:56.312355 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:56.313057 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:56.314418 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:56.315234 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:56.315812 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:56.315916 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:56.318582 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:56.318585 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:56.318586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:56.318597 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:56.320900 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:56.321344 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:56.321347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:56.321348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:56.321453 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:56.321671 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:56.321672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:56.321673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:56.321939 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:56.322083 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:56.322203 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 17:05:56.322294 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260808 17:05:56.323336 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:56.326596 1829 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:56.327271 1829 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:56.330406 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:56.324744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:47637 (local address 127.0.0.1:33759)
0808 17:05:56.324780 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:56.324782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:56.324790 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:05:56.324851 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:56.324852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:56.324861 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:56.324907 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:56.325477 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:56.326222 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:56.326923 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:56.327134 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:56.327185 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:56.328175 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:56.328177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:56.328177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:05:56.328184 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:56.329409 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:56.329730 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:56.329731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:56.329731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:56.329780 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:05:56.329959 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:05:56.329960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:05:56.329961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:05:56.330056 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:56.330152 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:56.330275 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 17:05:56.330329 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:05:56.332299 1829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e93fd66675854efd95f7840f1c705e67: Generated new TSK 0
I20260808 17:05:56.332459 1829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:56.336766 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 1686
W20260808 17:05:56.352411 1661 connection.cc:535] client connection to 127.0.0.1:33759 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33759): Connection reset by peer (error 104)
2026-08-08T17:05:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:56 dist-test-slave-2j7r krb5kdc[1864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:56 dist-test-slave-2j7r krb5kdc[1864](info): set up 2 sockets
Aug 08 17:05:56 dist-test-slave-2j7r krb5kdc[1864](info): commencing operation
krb5kdc: starting...
W20260808 17:05:57.495738 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:57 dist-test-slave-2j7r krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:57.605134 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34951
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:57.684613 1883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:57.684803 1883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:57.684824 1883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:57.685917 1883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:57.685971 1883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:57.685984 1883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:57.685995 1883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:57.686012 1883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:57.687404 1883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34951
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:57.687582 1883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:57.687742 1883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:57.692135 1893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:57.692143 1895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:57.692342 1883 server_base.cc:1034] running on GCE node
W20260808 17:05:57.692340 1892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:57.692612 1883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:57.692893 1883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:57.694018 1883 hybrid_clock.cc:648] HybridClock initialized: now 1786208757694009 us; error 27 us; skew 500 ppm
Aug 08 17:05:57 dist-test-slave-2j7r krb5kdc[1864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208757, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:57.700469 1883 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:57.700927 1883 webserver.cc:460] Webserver started at http://127.0.0.1:41143/ using document root <none> and password file <none>
I20260808 17:05:57.701171 1883 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:57.701234 1883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:57.701366 1883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:57.702371 1883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "c314864bf1d04e3ead572b3a1ef0d04a"
format_stamp: "Formatted at 2026-08-08 17:05:57 on dist-test-slave-2j7r"
server_key: "2fdbe9b2cc802d8f8fbbe9ccc19fac42"
server_key_iv: "ab8a176f5ad62b9a7e129d25567a2044"
server_key_version: "encryptionkey@0"
I20260808 17:05:57.702713 1883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "c314864bf1d04e3ead572b3a1ef0d04a"
format_stamp: "Formatted at 2026-08-08 17:05:57 on dist-test-slave-2j7r"
server_key: "2fdbe9b2cc802d8f8fbbe9ccc19fac42"
server_key_iv: "ab8a176f5ad62b9a7e129d25567a2044"
server_key_version: "encryptionkey@0"
I20260808 17:05:57.704000 1883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:05:57.704708 1905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:57.704854 1883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:05:57.704952 1883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "c314864bf1d04e3ead572b3a1ef0d04a"
format_stamp: "Formatted at 2026-08-08 17:05:57 on dist-test-slave-2j7r"
server_key: "2fdbe9b2cc802d8f8fbbe9ccc19fac42"
server_key_iv: "ab8a176f5ad62b9a7e129d25567a2044"
server_key_version: "encryptionkey@0"
I20260808 17:05:57.705016 1883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:57.714296 1883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:57.716336 1883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:57.716471 1883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:57.720439 1883 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34951
I20260808 17:05:57.720468 1998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34951 every 8 connection(s)
I20260808 17:05:57.720788 1883 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:57.721608 1999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:57.723680 1999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a: Bootstrap starting.
I20260808 17:05:57.724205 1999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:57.724476 1999 log.cc:824] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:57.725138 1999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a: No bootstrap required, opened a new log
I20260808 17:05:57.726656 1999 raft_consensus.cc:348] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c314864bf1d04e3ead572b3a1ef0d04a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260808 17:05:57.726818 1999 raft_consensus.cc:374] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:57.726866 1999 raft_consensus.cc:729] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c314864bf1d04e3ead572b3a1ef0d04a, State: Initialized, Role: FOLLOWER
I20260808 17:05:57.726998 1999 consensus_queue.cc:260] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c314864bf1d04e3ead572b3a1ef0d04a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260808 17:05:57.727080 1999 raft_consensus.cc:388] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:57.727113 1999 raft_consensus.cc:482] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:57.727159 1999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:57.727641 1999 raft_consensus.cc:504] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c314864bf1d04e3ead572b3a1ef0d04a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260808 17:05:57.727732 1999 leader_election.cc:302] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c314864bf1d04e3ead572b3a1ef0d04a; no voters:
I20260808 17:05:57.727880 1999 leader_election.cc:288] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:57.727922 2004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:57.728070 2004 raft_consensus.cc:686] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [term 1 LEADER]: Becoming Leader. State: Replica: c314864bf1d04e3ead572b3a1ef0d04a, State: Running, Role: LEADER
I20260808 17:05:57.728109 1999 sys_catalog.cc:564] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:57.728152 2004 consensus_queue.cc:237] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c314864bf1d04e3ead572b3a1ef0d04a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260808 17:05:57.728665 2008 sys_catalog.cc:455] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c314864bf1d04e3ead572b3a1ef0d04a. Latest consensus state: current_term: 1 leader_uuid: "c314864bf1d04e3ead572b3a1ef0d04a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c314864bf1d04e3ead572b3a1ef0d04a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } } }
I20260808 17:05:57.728761 2008 sys_catalog.cc:458] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:57.729074 2017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:57.729071 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 1883
I20260808 17:05:57.729126 2005 sys_catalog.cc:455] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c314864bf1d04e3ead572b3a1ef0d04a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c314864bf1d04e3ead572b3a1ef0d04a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } } }
I20260808 17:05:57.729194 2005 sys_catalog.cc:458] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:57.729264 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:57.729463 31496 external_mini_cluster.cc:1428] Setting key 05f1c398e6aa07a5a591c3e6ebb58668
I20260808 17:05:57.729812 2017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:57.731933 2017 catalog_manager.cc:1347] Generated new cluster ID: 95d5b7a131f546cd81fae488fabd89d3
I20260808 17:05:57.731976 2017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:57.734879 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:57.729873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:42393 (local address 127.0.0.1:34951)
0808 17:05:57.730035 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:57.730043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:57.730159 (+ 116us) server_negotiation.cc:407] Connection header received
0808 17:05:57.730408 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:57.730416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:57.730477 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:57.730613 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:57.730813 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:57.731832 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:57.732907 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:57.733528 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:57.733622 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:57.734228 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:57.734232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:57.734233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:57.734445 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:57.734479 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:57.734493 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:57.734566 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:57.734669 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 17:05:57.734692 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260808 17:05:57.735817 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:57.739221 2017 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:57.739879 2017 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:57.742121 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:57.737201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:48917 (local address 127.0.0.1:34951)
0808 17:05:57.737233 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:57.737234 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:57.739030 (+ 1796us) server_negotiation.cc:407] Connection header received
0808 17:05:57.739076 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:57.739077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:57.739088 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:57.739136 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:57.739360 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:57.740033 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:57.741009 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:05:57.741304 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:05:57.741354 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:05:57.741909 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:57.741910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:57.741910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:57.741914 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:57.741925 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:57.741929 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:57.741981 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:57.742045 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 17:05:57.742055 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:05:57.746445 2017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c314864bf1d04e3ead572b3a1ef0d04a: Generated new TSK 0
I20260808 17:05:57.746583 2017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:57.753676 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 1883
W20260808 17:05:57.789196 1859 connection.cc:535] client connection to 127.0.0.1:34951 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34951): Connection reset by peer (error 104)
2026-08-08T17:05:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:57 dist-test-slave-2j7r krb5kdc[2043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:57 dist-test-slave-2j7r krb5kdc[2043](info): set up 2 sockets
Aug 08 17:05:57 dist-test-slave-2j7r krb5kdc[2043](info): commencing operation
krb5kdc: starting...
W20260808 17:05:58.920842 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:05:58 dist-test-slave-2j7r krb5kdc[2043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:05:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:05:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:05:59.029623 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42837
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:05:59.107933 2063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:05:59.108110 2063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:05:59.108131 2063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:05:59.109191 2063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:05:59.109230 2063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:05:59.109244 2063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:05:59.109257 2063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:05:59.109268 2063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:05:59.110622 2063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42837
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:05:59.110790 2063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:05:59.110944 2063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:05:59.115190 2081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:59.115235 2077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:05:59.115231 2075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:05:59.115414 2063 server_base.cc:1034] running on GCE node
I20260808 17:05:59.115617 2063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:05:59.115872 2063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:05:59.117015 2063 hybrid_clock.cc:648] HybridClock initialized: now 1786208759117004 us; error 24 us; skew 500 ppm
Aug 08 17:05:59 dist-test-slave-2j7r krb5kdc[2043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208759, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:05:59.119205 2063 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:05:59.119578 2063 webserver.cc:460] Webserver started at http://127.0.0.1:37945/ using document root <none> and password file <none>
I20260808 17:05:59.119760 2063 fs_manager.cc:359] Metadata directory not provided
I20260808 17:05:59.119804 2063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:05:59.119922 2063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:05:59.120827 2063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "4e19e8b758fd4ddc8c58cccbb48bd480"
format_stamp: "Formatted at 2026-08-08 17:05:59 on dist-test-slave-2j7r"
server_key: "4e54fe630a9668a172268dc0506761de"
server_key_iv: "5abc3ac2413a96b3a965ecc5cb7616f6"
server_key_version: "encryptionkey@0"
I20260808 17:05:59.121187 2063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "4e19e8b758fd4ddc8c58cccbb48bd480"
format_stamp: "Formatted at 2026-08-08 17:05:59 on dist-test-slave-2j7r"
server_key: "4e54fe630a9668a172268dc0506761de"
server_key_iv: "5abc3ac2413a96b3a965ecc5cb7616f6"
server_key_version: "encryptionkey@0"
I20260808 17:05:59.122367 2063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:05:59.122991 2089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:05:59.123124 2063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:05:59.123204 2063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "4e19e8b758fd4ddc8c58cccbb48bd480"
format_stamp: "Formatted at 2026-08-08 17:05:59 on dist-test-slave-2j7r"
server_key: "4e54fe630a9668a172268dc0506761de"
server_key_iv: "5abc3ac2413a96b3a965ecc5cb7616f6"
server_key_version: "encryptionkey@0"
I20260808 17:05:59.123292 2063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:05:59.126353 2063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:05:59.128180 2063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:05:59.128302 2063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:05:59.131795 2063 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42837
I20260808 17:05:59.131824 2170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42837 every 8 connection(s)
I20260808 17:05:59.132090 2063 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:05:59.132843 2171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:05:59.133078 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 2063
I20260808 17:05:59.133239 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:05:59.133395 31496 external_mini_cluster.cc:1428] Setting key 647ed44920bc428b580ca7ea7a4d4bf4
I20260808 17:05:59.135529 2171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480: Bootstrap starting.
I20260808 17:05:59.135560 2176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:59.133837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:56671 (local address 127.0.0.1:42837)
0808 17:05:59.134014 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:59.134023 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:59.134054 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:05:59.134530 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:59.134539 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:59.134572 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:59.134714 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:59.135179 (+ 465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:59.135182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:59.135183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:59.135186 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:59.135227 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:59.135240 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:59.135317 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:59.135356 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 17:05:59.135394 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
I20260808 17:05:59.136204 2171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480: Neither blocks nor log segments found. Creating new log.
I20260808 17:05:59.136487 2171 log.cc:824] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480: Log is configured to *not* fsync() on all Append() calls
I20260808 17:05:59.136503 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:05:59.142750 2171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480: No bootstrap required, opened a new log
I20260808 17:05:59.144550 2171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e19e8b758fd4ddc8c58cccbb48bd480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42837 } }
I20260808 17:05:59.144682 2176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:05:59.141702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:40059 (local address 127.0.0.1:42837)
0808 17:05:59.142456 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0808 17:05:59.142458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:05:59.142557 (+ 99us) server_negotiation.cc:407] Connection header received
0808 17:05:59.142643 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:05:59.142644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:05:59.142649 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:05:59.142673 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:05:59.144491 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:05:59.144493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:05:59.144493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:05:59.144498 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:05:59.144516 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:05:59.144520 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:05:59.144579 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:05:59.144622 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 17:05:59.144633 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731}
I20260808 17:05:59.144718 2171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:05:59.144754 2171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e19e8b758fd4ddc8c58cccbb48bd480, State: Initialized, Role: FOLLOWER
I20260808 17:05:59.144891 2171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e19e8b758fd4ddc8c58cccbb48bd480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42837 } }
I20260808 17:05:59.144970 2171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:05:59.144995 2171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:05:59.145046 2171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:05:59.145560 2171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e19e8b758fd4ddc8c58cccbb48bd480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42837 } }
I20260808 17:05:59.145643 2171 leader_election.cc:302] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e19e8b758fd4ddc8c58cccbb48bd480; no voters:
I20260808 17:05:59.145752 2171 leader_election.cc:288] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:05:59.145795 2189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:05:59.145941 2189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [term 1 LEADER]: Becoming Leader. State: Replica: 4e19e8b758fd4ddc8c58cccbb48bd480, State: Running, Role: LEADER
I20260808 17:05:59.145977 2171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:05:59.146051 2189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e19e8b758fd4ddc8c58cccbb48bd480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42837 } }
I20260808 17:05:59.146399 2191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e19e8b758fd4ddc8c58cccbb48bd480. Latest consensus state: current_term: 1 leader_uuid: "4e19e8b758fd4ddc8c58cccbb48bd480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e19e8b758fd4ddc8c58cccbb48bd480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42837 } } }
I20260808 17:05:59.146518 2191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:59.146392 2190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e19e8b758fd4ddc8c58cccbb48bd480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e19e8b758fd4ddc8c58cccbb48bd480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42837 } } }
I20260808 17:05:59.146680 2190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480 [sys.catalog]: This master's current role is: LEADER
I20260808 17:05:59.147006 2201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:05:59.147475 2201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:05:59.148810 2201 catalog_manager.cc:1347] Generated new cluster ID: 0fe774f8d98b46ec9b5fa72c4d5932a2
I20260808 17:05:59.148862 2201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:05:59.154544 2201 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:05:59.154675 2201 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:05:59.159402 2201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e19e8b758fd4ddc8c58cccbb48bd480: Generated new TSK 0
I20260808 17:05:59.159538 2201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:05:59.164974 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 2063
2026-08-08T17:05:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:05:59 dist-test-slave-2j7r krb5kdc[2229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:05:59 dist-test-slave-2j7r krb5kdc[2229](info): set up 2 sockets
Aug 08 17:05:59 dist-test-slave-2j7r krb5kdc[2229](info): commencing operation
krb5kdc: starting...
W20260808 17:06:00.310272 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:06:00 dist-test-slave-2j7r krb5kdc[2229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:06:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:06:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:06:00.417516 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:37769
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:37997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:37769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:06:00.497285 2251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:00.497485 2251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:00.497519 2251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:00.498641 2251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:00.498687 2251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:00.498710 2251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:00.498729 2251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:00.498749 2251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:00.500214 2251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:37997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:37769
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:37769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:00.500459 2251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:00.500669 2251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:00.505187 2257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:00.505196 2261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:00.505379 2251 server_base.cc:1034] running on GCE node
W20260808 17:06:00.505234 2258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:00.505674 2251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:00.505975 2251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:00.507138 2251 hybrid_clock.cc:648] HybridClock initialized: now 1786208760507118 us; error 30 us; skew 500 ppm
Aug 08 17:06:00 dist-test-slave-2j7r krb5kdc[2229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208760, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:00.509641 2251 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:06:00.510058 2251 webserver.cc:460] Webserver started at http://127.30.194.62:39669/ using document root <none> and password file <none>
I20260808 17:06:00.510251 2251 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:00.510293 2251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:00.510386 2251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:00.511333 2251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "370886a5d8744f0899d13be3cc552526"
format_stamp: "Formatted at 2026-08-08 17:06:00 on dist-test-slave-2j7r"
server_key: "785e81306927a5c3396b04c85aba9228"
server_key_iv: "e4f518c14c18de078ad307ca1e682ee6"
server_key_version: "encryptionkey@0"
I20260808 17:06:00.511632 2251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "370886a5d8744f0899d13be3cc552526"
format_stamp: "Formatted at 2026-08-08 17:06:00 on dist-test-slave-2j7r"
server_key: "785e81306927a5c3396b04c85aba9228"
server_key_iv: "e4f518c14c18de078ad307ca1e682ee6"
server_key_version: "encryptionkey@0"
I20260808 17:06:00.512836 2251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:06:00.513568 2279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:00.513721 2251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:00.513805 2251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "370886a5d8744f0899d13be3cc552526"
format_stamp: "Formatted at 2026-08-08 17:06:00 on dist-test-slave-2j7r"
server_key: "785e81306927a5c3396b04c85aba9228"
server_key_iv: "e4f518c14c18de078ad307ca1e682ee6"
server_key_version: "encryptionkey@0"
I20260808 17:06:00.513856 2251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:00.524506 2251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:00.526146 2251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:00.526243 2251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:00.529733 2251 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:37769
I20260808 17:06:00.529775 2371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:37769 every 8 connection(s)
I20260808 17:06:00.530046 2251 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:06:00.530778 2372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:00.531525 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 2251
I20260808 17:06:00.531625 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:06:00.531780 31496 external_mini_cluster.cc:1428] Setting key 5274ab1a430d8fe913412ee27090b802
I20260808 17:06:00.533571 2372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526: Bootstrap starting.
I20260808 17:06:00.534248 2372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:00.534548 2372 log.cc:824] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:00.535238 2372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526: No bootstrap required, opened a new log
I20260808 17:06:00.537243 2372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370886a5d8744f0899d13be3cc552526" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37769 } }
I20260808 17:06:00.537364 2372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:06:00.537388 2372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 370886a5d8744f0899d13be3cc552526, State: Initialized, Role: FOLLOWER
I20260808 17:06:00.537497 2372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370886a5d8744f0899d13be3cc552526" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37769 } }
I20260808 17:06:00.537559 2372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:06:00.537578 2372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:06:00.537621 2372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 17:06:00 dist-test-slave-2j7r krb5kdc[2229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:06:00.538120 2372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370886a5d8744f0899d13be3cc552526" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37769 } }
I20260808 17:06:00.538206 2372 leader_election.cc:302] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 370886a5d8744f0899d13be3cc552526; no voters:
I20260808 17:06:00.538350 2372 leader_election.cc:288] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:06:00.538406 2378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:06:00.538606 2372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:00.538527 2378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [term 1 LEADER]: Becoming Leader. State: Replica: 370886a5d8744f0899d13be3cc552526, State: Running, Role: LEADER
I20260808 17:06:00.538905 2378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370886a5d8744f0899d13be3cc552526" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37769 } }
I20260808 17:06:00.539260 2378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 370886a5d8744f0899d13be3cc552526. Latest consensus state: current_term: 1 leader_uuid: "370886a5d8744f0899d13be3cc552526" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370886a5d8744f0899d13be3cc552526" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37769 } } }
I20260808 17:06:00.539395 2381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "370886a5d8744f0899d13be3cc552526" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370886a5d8744f0899d13be3cc552526" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 37769 } } }
I20260808 17:06:00.539456 2378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:00.540113 2381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526 [sys.catalog]: This master's current role is: LEADER
W20260808 17:06:00.540355 2397 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:06:00.540459 2397 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:06:00.540534 2402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:06:00.540693 2402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:06:00.542132 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:00.532217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58061 (local address 127.30.194.62:37769)
0808 17:06:00.532375 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:00.532394 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:00.532425 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:06:00.532841 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:00.532850 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:00.532914 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:00.533053 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:00.533640 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:00.534884 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:00.535631 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:00.536236 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:00.536347 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:00.538497 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:00.538501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:00.538502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:00.538513 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:00.540351 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:00.540807 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:00.540811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:00.540811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:00.540910 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:00.541166 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:00.541167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:00.541167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:00.541455 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:00.541595 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:00.541862 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 17:06:00.541969 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260808 17:06:00.542268 2402 catalog_manager.cc:1347] Generated new cluster ID: 3fa479aaee1943fb9b76a42ee3fb1ef2
I20260808 17:06:00.542311 2402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:06:00.542987 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:06:00.550927 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:00.544333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49057 (local address 127.30.194.62:37769)
0808 17:06:00.544363 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:00.544364 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:00.544480 (+ 116us) server_negotiation.cc:407] Connection header received
0808 17:06:00.544539 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:00.544539 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:00.544547 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:00.544593 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:00.545213 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:00.545897 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:00.546860 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:00.547179 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:00.547232 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:00.548470 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:00.548471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:00.548471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:00.548476 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:00.549858 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:00.550235 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:00.550236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:00.550236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:00.550282 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:00.550474 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:00.550475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:00.550475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:00.550575 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:00.550654 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:00.550783 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 17:06:00.550836 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 17:06:00.570987 2402 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:06:00.571467 2402 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:06:00.579834 2402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 370886a5d8744f0899d13be3cc552526: Generated new TSK 0
I20260808 17:06:00.579991 2402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:06:00.586993 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 2251
W20260808 17:06:00.599467 2226 connection.cc:535] client connection to 127.30.194.62:37769 recv error: Network error: failed to read from TLS socket (remote: 127.30.194.62:37769): Connection reset by peer (error 104)
2026-08-08T17:06:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:06:00 dist-test-slave-2j7r krb5kdc[2416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:06:00 dist-test-slave-2j7r krb5kdc[2416](info): set up 2 sockets
Aug 08 17:06:00 dist-test-slave-2j7r krb5kdc[2416](info): commencing operation
krb5kdc: starting...
W20260808 17:06:01.752315 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:06:01 dist-test-slave-2j7r krb5kdc[2416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:06:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:06:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:06:01.860781 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:38739
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:38739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:06:01.940826 2437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:01.941004 2437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:01.941025 2437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:01.942114 2437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:01.942155 2437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:01.942168 2437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:01.942179 2437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:01.942189 2437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:01.943563 2437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:38739
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:38739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:01.943744 2437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:01.943893 2437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:01.948279 2445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:01.948289 2446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:01.948289 2448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:01.948484 2437 server_base.cc:1034] running on GCE node
I20260808 17:06:01.948702 2437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:01.948973 2437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:01.950109 2437 hybrid_clock.cc:648] HybridClock initialized: now 1786208761950095 us; error 31 us; skew 500 ppm
Aug 08 17:06:01 dist-test-slave-2j7r krb5kdc[2416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208761, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:01.952261 2437 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:06:01.952726 2437 webserver.cc:460] Webserver started at http://127.30.194.62:40153/ using document root <none> and password file <none>
I20260808 17:06:01.952973 2437 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:01.953034 2437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:01.953186 2437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:01.954469 2437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "76ef15dc4c26409da4cd82a77edfdf03"
format_stamp: "Formatted at 2026-08-08 17:06:01 on dist-test-slave-2j7r"
server_key: "f3d040bd4f054d0ff1b542ca58be0b5c"
server_key_iv: "116c79598ae2ccd0b7093512134b4006"
server_key_version: "encryptionkey@0"
I20260808 17:06:01.954823 2437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "76ef15dc4c26409da4cd82a77edfdf03"
format_stamp: "Formatted at 2026-08-08 17:06:01 on dist-test-slave-2j7r"
server_key: "f3d040bd4f054d0ff1b542ca58be0b5c"
server_key_iv: "116c79598ae2ccd0b7093512134b4006"
server_key_version: "encryptionkey@0"
I20260808 17:06:01.956153 2437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:06:01.956794 2457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:01.956934 2437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:06:01.957015 2437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "76ef15dc4c26409da4cd82a77edfdf03"
format_stamp: "Formatted at 2026-08-08 17:06:01 on dist-test-slave-2j7r"
server_key: "f3d040bd4f054d0ff1b542ca58be0b5c"
server_key_iv: "116c79598ae2ccd0b7093512134b4006"
server_key_version: "encryptionkey@0"
I20260808 17:06:01.957078 2437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:01.987548 2437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:01.989132 2437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:01.989259 2437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:01.992831 2437 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:38739
I20260808 17:06:01.992864 2545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:38739 every 8 connection(s)
I20260808 17:06:01.993175 2437 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:06:01.993958 2546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:01.996057 2546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03: Bootstrap starting.
I20260808 17:06:01.996605 2546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:01.996595 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 2437
I20260808 17:06:01.996693 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:06:01.996863 31496 external_mini_cluster.cc:1428] Setting key d9fa6a97652f6725db9f68e072942176
I20260808 17:06:01.996871 2546 log.cc:824] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:01.997635 2546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03: No bootstrap required, opened a new log
I20260808 17:06:02.000123 2546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ef15dc4c26409da4cd82a77edfdf03" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38739 } }
I20260808 17:06:02.000250 2546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:06:02.000280 2546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ef15dc4c26409da4cd82a77edfdf03, State: Initialized, Role: FOLLOWER
I20260808 17:06:02.000406 2546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ef15dc4c26409da4cd82a77edfdf03" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38739 } }
I20260808 17:06:02.000475 2546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:06:02.000506 2546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:06:02.000551 2546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:06:02.001053 2546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ef15dc4c26409da4cd82a77edfdf03" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38739 } }
I20260808 17:06:02.001178 2546 leader_election.cc:302] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76ef15dc4c26409da4cd82a77edfdf03; no voters:
I20260808 17:06:02.001325 2546 leader_election.cc:288] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:06:02.001365 2552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:06:02.001477 2552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [term 1 LEADER]: Becoming Leader. State: Replica: 76ef15dc4c26409da4cd82a77edfdf03, State: Running, Role: LEADER
I20260808 17:06:02.001585 2546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:02.001608 2552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ef15dc4c26409da4cd82a77edfdf03" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38739 } }
I20260808 17:06:02.002074 2555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76ef15dc4c26409da4cd82a77edfdf03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ef15dc4c26409da4cd82a77edfdf03" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38739 } } }
I20260808 17:06:02.002171 2555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:02.002063 2553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76ef15dc4c26409da4cd82a77edfdf03. Latest consensus state: current_term: 1 leader_uuid: "76ef15dc4c26409da4cd82a77edfdf03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ef15dc4c26409da4cd82a77edfdf03" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38739 } } }
I20260808 17:06:02.002298 2553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03 [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:02.002478 2571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:06:02.002497 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:01.997328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36431 (local address 127.30.194.62:38739)
0808 17:06:01.997497 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:01.997506 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:01.997536 (+ 30us) server_negotiation.cc:407] Connection header received
0808 17:06:01.998020 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:01.998029 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:01.998089 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:01.998231 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:01.998449 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:01.999783 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:02.000582 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:02.000986 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:02.001099 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:02.001691 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:02.001695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:02.001695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:06:02.001880 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:02.001912 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:06:02.001923 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:02.002002 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:02.002272 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 17:06:02.002304 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260808 17:06:02.003273 2571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:06:02.003486 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:06:02.004730 2571 catalog_manager.cc:1347] Generated new cluster ID: 31ff622f8f7f4f76b74992272503d980
I20260808 17:06:02.004776 2571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:06:02.008967 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:02.005400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44605 (local address 127.30.194.62:38739)
0808 17:06:02.005440 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:02.005441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:02.005590 (+ 149us) server_negotiation.cc:407] Connection header received
0808 17:06:02.005678 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:02.005679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:02.005689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:02.005715 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:02.005980 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:02.006674 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:02.007717 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:02.008035 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:02.008076 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:02.008732 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:02.008733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:02.008733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:06:02.008737 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:02.008750 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:06:02.008754 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:02.008810 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:02.008890 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 17:06:02.008899 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:06:02.010779 2571 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:06:02.011232 2571 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:06:02.017762 2571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76ef15dc4c26409da4cd82a77edfdf03: Generated new TSK 0
I20260808 17:06:02.017901 2571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:06:02.026367 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 2437
W20260808 17:06:02.041930 2413 connection.cc:535] client connection to 127.30.194.62:38739 recv error: Network error: failed to read from TLS socket (remote: 127.30.194.62:38739): Connection reset by peer (error 104)
2026-08-08T17:06:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:06:02 dist-test-slave-2j7r krb5kdc[2599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:06:02 dist-test-slave-2j7r krb5kdc[2599](info): set up 2 sockets
Aug 08 17:06:02 dist-test-slave-2j7r krb5kdc[2599](info): commencing operation
krb5kdc: starting...
W20260808 17:06:03.176191 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:06:03 dist-test-slave-2j7r krb5kdc[2599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:06:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:06:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:06:03.283334 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:33961
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:33651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:33961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:06:03.360898 2624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:03.361100 2624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:03.361133 2624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:03.362234 2624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:03.362272 2624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:03.362291 2624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:03.362306 2624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:03.362322 2624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:03.363730 2624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:33651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:33961
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:33961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:03.363960 2624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:03.364174 2624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:03.368602 2632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:03.368616 2631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:03.368760 2624 server_base.cc:1034] running on GCE node
W20260808 17:06:03.368639 2635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:03.369052 2624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:03.369347 2624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:03.370497 2624 hybrid_clock.cc:648] HybridClock initialized: now 1786208763370479 us; error 35 us; skew 500 ppm
Aug 08 17:06:03 dist-test-slave-2j7r krb5kdc[2599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208763, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:03.375782 2624 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:06:03.376180 2624 webserver.cc:460] Webserver started at http://127.30.194.62:42729/ using document root <none> and password file <none>
I20260808 17:06:03.376370 2624 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:03.376415 2624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:03.376514 2624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:03.377521 2624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "bb76412ac13845b4b9b0798cbbf37726"
format_stamp: "Formatted at 2026-08-08 17:06:03 on dist-test-slave-2j7r"
server_key: "f97cc4ea793ffaba6f06e30c65af5c98"
server_key_iv: "4029568e27ebb856587136f628b3bfc7"
server_key_version: "encryptionkey@0"
I20260808 17:06:03.377822 2624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "bb76412ac13845b4b9b0798cbbf37726"
format_stamp: "Formatted at 2026-08-08 17:06:03 on dist-test-slave-2j7r"
server_key: "f97cc4ea793ffaba6f06e30c65af5c98"
server_key_iv: "4029568e27ebb856587136f628b3bfc7"
server_key_version: "encryptionkey@0"
I20260808 17:06:03.378975 2624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:06:03.379611 2649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:03.379768 2624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:06:03.379850 2624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "bb76412ac13845b4b9b0798cbbf37726"
format_stamp: "Formatted at 2026-08-08 17:06:03 on dist-test-slave-2j7r"
server_key: "f97cc4ea793ffaba6f06e30c65af5c98"
server_key_iv: "4029568e27ebb856587136f628b3bfc7"
server_key_version: "encryptionkey@0"
I20260808 17:06:03.379901 2624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:03.382917 2624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:03.384758 2624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:03.384855 2624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:03.388337 2624 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:33961
I20260808 17:06:03.388397 2743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:33961 every 8 connection(s)
I20260808 17:06:03.388633 2624 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:06:03.389556 2747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:03.391515 2747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726: Bootstrap starting.
I20260808 17:06:03.392028 2747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:03.392266 2747 log.cc:824] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:03.392901 2747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726: No bootstrap required, opened a new log
I20260808 17:06:03.394411 2747 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb76412ac13845b4b9b0798cbbf37726" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33961 } }
I20260808 17:06:03.394532 2747 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:06:03.394552 2747 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb76412ac13845b4b9b0798cbbf37726, State: Initialized, Role: FOLLOWER
I20260808 17:06:03.394666 2747 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb76412ac13845b4b9b0798cbbf37726" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33961 } }
I20260808 17:06:03.394723 2747 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:06:03.394743 2747 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:06:03.394788 2747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:06:03.395525 2747 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb76412ac13845b4b9b0798cbbf37726" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33961 } }
I20260808 17:06:03.395609 2747 leader_election.cc:302] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb76412ac13845b4b9b0798cbbf37726; no voters:
I20260808 17:06:03.395752 2747 leader_election.cc:288] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:06:03.395800 2758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:06:03.395982 2747 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:03.395988 2758 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [term 1 LEADER]: Becoming Leader. State: Replica: bb76412ac13845b4b9b0798cbbf37726, State: Running, Role: LEADER
I20260808 17:06:03.396214 2758 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb76412ac13845b4b9b0798cbbf37726" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33961 } }
I20260808 17:06:03.396544 2761 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb76412ac13845b4b9b0798cbbf37726. Latest consensus state: current_term: 1 leader_uuid: "bb76412ac13845b4b9b0798cbbf37726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb76412ac13845b4b9b0798cbbf37726" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33961 } } }
I20260808 17:06:03.396637 2761 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:03.396551 2759 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb76412ac13845b4b9b0798cbbf37726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb76412ac13845b4b9b0798cbbf37726" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 33961 } } }
I20260808 17:06:03.396872 2759 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726 [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:03.396857 2778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:06:03.397135 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 2624
I20260808 17:06:03.397229 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:06:03.397444 31496 external_mini_cluster.cc:1428] Setting key d356eec05315d090452cc9264f8576b2
I20260808 17:06:03.397612 2778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:06:03.399312 2778 catalog_manager.cc:1347] Generated new cluster ID: e31cb83bf7184a289c779b960704d24e
I20260808 17:06:03.399363 2778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:06:03.402540 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:03.398055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60371 (local address 127.30.194.62:33961)
0808 17:06:03.401243 (+ 3188us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:03.401252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:03.401288 (+ 36us) server_negotiation.cc:407] Connection header received
0808 17:06:03.401397 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:03.401403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:03.401431 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:03.401579 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:03.402154 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:03.402158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:03.402158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:06:03.402163 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:03.402205 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:06:03.402220 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:03.402296 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:03.402333 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 17:06:03.402372 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3121,"thread_start_us":67,"threads_started":1}
I20260808 17:06:03.403475 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:06:03.405908 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:03.404888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48895 (local address 127.30.194.62:33961)
0808 17:06:03.404925 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:03.404927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:03.405072 (+ 145us) server_negotiation.cc:407] Connection header received
0808 17:06:03.405157 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:03.405158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:03.405163 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:03.405226 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:03.405712 (+ 486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:03.405713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:03.405713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:06:03.405718 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:03.405732 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:06:03.405736 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:03.405800 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:03.405837 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 17:06:03.405850 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:06:03.409298 2778 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:06:03.409412 2778 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:06:03.417079 2778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb76412ac13845b4b9b0798cbbf37726: Generated new TSK 0
I20260808 17:06:03.417238 2778 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:06:03.423563 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 2624
2026-08-08T17:06:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:06:03 dist-test-slave-2j7r krb5kdc[2808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:06:03 dist-test-slave-2j7r krb5kdc[2808](info): set up 2 sockets
Aug 08 17:06:03 dist-test-slave-2j7r krb5kdc[2808](info): commencing operation
krb5kdc: starting...
W20260808 17:06:04.613132 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:06:04 dist-test-slave-2j7r krb5kdc[2808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:06:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:06:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:06:04.721741 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:38469
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:38469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:06:04.799751 2832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:04.799927 2832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:04.799948 2832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:04.800987 2832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:04.801019 2832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:04.801033 2832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:04.801043 2832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:04.801054 2832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:04.802479 2832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:38469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:38469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:04.802657 2832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:04.802819 2832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:04.807396 2841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:04.807448 2832 server_base.cc:1034] running on GCE node
W20260808 17:06:04.807380 2840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:04.807369 2843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:04.807755 2832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:04.807999 2832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:04.809131 2832 hybrid_clock.cc:648] HybridClock initialized: now 1786208764809123 us; error 27 us; skew 500 ppm
Aug 08 17:06:04 dist-test-slave-2j7r krb5kdc[2808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208764, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:04.811322 2832 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:06:04.811754 2832 webserver.cc:460] Webserver started at http://127.30.194.62:43861/ using document root <none> and password file <none>
I20260808 17:06:04.812007 2832 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:04.812067 2832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:04.812196 2832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:04.813321 2832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "a2c46612efe246118fa1aa1b9364c25e"
format_stamp: "Formatted at 2026-08-08 17:06:04 on dist-test-slave-2j7r"
server_key: "dc9ae1842525488d0435b81e8770604e"
server_key_iv: "2e9cefe73bc3e480a339fc79879903d0"
server_key_version: "encryptionkey@0"
I20260808 17:06:04.813634 2832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "a2c46612efe246118fa1aa1b9364c25e"
format_stamp: "Formatted at 2026-08-08 17:06:04 on dist-test-slave-2j7r"
server_key: "dc9ae1842525488d0435b81e8770604e"
server_key_iv: "2e9cefe73bc3e480a339fc79879903d0"
server_key_version: "encryptionkey@0"
I20260808 17:06:04.814831 2832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:06:04.815514 2860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:04.815655 2832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:06:04.815735 2832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "a2c46612efe246118fa1aa1b9364c25e"
format_stamp: "Formatted at 2026-08-08 17:06:04 on dist-test-slave-2j7r"
server_key: "dc9ae1842525488d0435b81e8770604e"
server_key_iv: "2e9cefe73bc3e480a339fc79879903d0"
server_key_version: "encryptionkey@0"
I20260808 17:06:04.815794 2832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:04.826632 2832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:04.828167 2832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:04.828289 2832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:04.831984 2832 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:38469
I20260808 17:06:04.832020 2952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:38469 every 8 connection(s)
I20260808 17:06:04.832304 2832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:06:04.833119 2953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:04.835323 2953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e: Bootstrap starting.
I20260808 17:06:04.835593 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 2832
I20260808 17:06:04.835690 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:06:04.835845 31496 external_mini_cluster.cc:1428] Setting key f6b0cbae0f0f62a72e1f9234ad5a4a64
I20260808 17:06:04.835939 2953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:04.836282 2953 log.cc:824] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:04.836961 2953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e: No bootstrap required, opened a new log
I20260808 17:06:04.839227 2953 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c46612efe246118fa1aa1b9364c25e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38469 } }
I20260808 17:06:04.839397 2953 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:06:04.839432 2953 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c46612efe246118fa1aa1b9364c25e, State: Initialized, Role: FOLLOWER
I20260808 17:06:04.839555 2953 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c46612efe246118fa1aa1b9364c25e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38469 } }
I20260808 17:06:04.839627 2953 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:06:04.839663 2953 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:06:04.839710 2953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:06:04.840267 2953 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c46612efe246118fa1aa1b9364c25e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38469 } }
I20260808 17:06:04.840379 2953 leader_election.cc:302] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2c46612efe246118fa1aa1b9364c25e; no voters:
I20260808 17:06:04.840546 2953 leader_election.cc:288] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:06:04.840566 2960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:06:04.840685 2960 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [term 1 LEADER]: Becoming Leader. State: Replica: a2c46612efe246118fa1aa1b9364c25e, State: Running, Role: LEADER
I20260808 17:06:04.840808 2960 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c46612efe246118fa1aa1b9364c25e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38469 } }
I20260808 17:06:04.840823 2953 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:04.841746 2965 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2c46612efe246118fa1aa1b9364c25e. Latest consensus state: current_term: 1 leader_uuid: "a2c46612efe246118fa1aa1b9364c25e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c46612efe246118fa1aa1b9364c25e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38469 } } }
I20260808 17:06:04.841831 2965 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:04.842025 2962 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2c46612efe246118fa1aa1b9364c25e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c46612efe246118fa1aa1b9364c25e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 38469 } } }
I20260808 17:06:04.842428 2962 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e [sys.catalog]: This master's current role is: LEADER
Aug 08 17:06:04 dist-test-slave-2j7r krb5kdc[2808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
W20260808 17:06:04.842908 2989 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:06:04.842989 2989 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:06:04.843075 2991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:06:04.843326 2991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:06:04.844831 2991 catalog_manager.cc:1347] Generated new cluster ID: da991c55cc534609a83751cab3420098
I20260808 17:06:04.844880 2991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:06:04.847142 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:04.836305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49931 (local address 127.30.194.62:38469)
0808 17:06:04.836487 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:04.836496 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:04.836534 (+ 38us) server_negotiation.cc:407] Connection header received
0808 17:06:04.836991 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:04.836999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:04.837061 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:04.837204 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:04.838000 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:04.838846 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:04.839876 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:04.840260 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:04.840332 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:04.843271 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:04.843275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:04.843275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:04.843286 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:04.845188 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:04.845590 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:04.845593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:04.845593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:04.845695 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:04.845893 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:04.845894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:04.845894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:04.846196 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:04.846346 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:04.846806 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 17:06:04.846938 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260808 17:06:04.848026 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:06:04.852262 2991 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:06:04.852916 2991 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:06:04.858506 2991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2c46612efe246118fa1aa1b9364c25e: Generated new TSK 0
I20260808 17:06:04.858695 2991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:06:04.859091 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:04.849511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33177 (local address 127.30.194.62:38469)
0808 17:06:04.849542 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:04.849543 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:04.852025 (+ 2482us) server_negotiation.cc:407] Connection header received
0808 17:06:04.852087 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:04.852088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:04.852097 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:04.852152 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:04.852825 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:04.853582 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:04.854535 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:04.854858 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:04.854914 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:04.856183 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:04.856185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:04.856185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:04.856192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:04.857665 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:04.858132 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:04.858134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:04.858134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:04.858192 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:04.858542 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:04.858543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:04.858544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:04.858680 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:04.858773 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:04.858925 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 17:06:04.859006 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 17:06:04.860903 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 2832
2026-08-08T17:06:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:06:04 dist-test-slave-2j7r krb5kdc[3016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:06:04 dist-test-slave-2j7r krb5kdc[3016](info): set up 2 sockets
Aug 08 17:06:04 dist-test-slave-2j7r krb5kdc[3016](info): commencing operation
krb5kdc: starting...
W20260808 17:06:06.052239 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:06:06 dist-test-slave-2j7r krb5kdc[3016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:06:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:06:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:06:06.160033 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43269
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:43269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:06:06.238736 3035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:06.238924 3035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:06.238945 3035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:06.240041 3035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:06.240080 3035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:06.240093 3035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:06.240115 3035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:06.240137 3035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:06.241606 3035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:43269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:06.241819 3035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:06.242008 3035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:06.246577 3054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:06.246762 3035 server_base.cc:1034] running on GCE node
W20260808 17:06:06.246578 3051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:06.246598 3047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:06.247080 3035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:06.247359 3035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:06.248508 3035 hybrid_clock.cc:648] HybridClock initialized: now 1786208766248496 us; error 26 us; skew 500 ppm
Aug 08 17:06:06 dist-test-slave-2j7r krb5kdc[3016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208766, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:06.250957 3035 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:06:06.251396 3035 webserver.cc:460] Webserver started at http://127.30.194.62:39227/ using document root <none> and password file <none>
I20260808 17:06:06.251587 3035 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:06.251633 3035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:06.251773 3035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:06.252864 3035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "3a5606a12af8476db1bf8ec53c7b9a7e"
format_stamp: "Formatted at 2026-08-08 17:06:06 on dist-test-slave-2j7r"
server_key: "24eb2f48f7b1041890d65abd9231bf49"
server_key_iv: "ad3d5907fd012ad160b90354c14b84d0"
server_key_version: "encryptionkey@0"
I20260808 17:06:06.253227 3035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "3a5606a12af8476db1bf8ec53c7b9a7e"
format_stamp: "Formatted at 2026-08-08 17:06:06 on dist-test-slave-2j7r"
server_key: "24eb2f48f7b1041890d65abd9231bf49"
server_key_iv: "ad3d5907fd012ad160b90354c14b84d0"
server_key_version: "encryptionkey@0"
I20260808 17:06:06.254549 3035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:06:06.255254 3063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:06.255407 3035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:06:06.255491 3035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "3a5606a12af8476db1bf8ec53c7b9a7e"
format_stamp: "Formatted at 2026-08-08 17:06:06 on dist-test-slave-2j7r"
server_key: "24eb2f48f7b1041890d65abd9231bf49"
server_key_iv: "ad3d5907fd012ad160b90354c14b84d0"
server_key_version: "encryptionkey@0"
I20260808 17:06:06.255544 3035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:06.267381 3035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:06.269277 3035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:06.269382 3035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:06.272809 3035 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:43269
I20260808 17:06:06.272841 3169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:43269 every 8 connection(s)
I20260808 17:06:06.273152 3035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:06:06.273873 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 3035
I20260808 17:06:06.273993 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:06:06.274039 3170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:06.274184 31496 external_mini_cluster.cc:1428] Setting key 0ec10562dd9b2e32bafc7097b81b9563
I20260808 17:06:06.276760 3170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e: Bootstrap starting.
I20260808 17:06:06.277444 3170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:06.277724 3170 log.cc:824] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:06.278414 3170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e: No bootstrap required, opened a new log
I20260808 17:06:06.279314 3176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:06.274716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48795 (local address 127.30.194.62:43269)
0808 17:06:06.274868 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:06.274875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:06.274898 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:06:06.275319 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:06.275325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:06.275368 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:06.275471 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:06.275689 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:06.276524 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:06.277591 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:06.278004 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:06.278079 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:06.278677 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:06.278682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:06.278682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:06:06.278867 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:06.278904 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:06:06.278918 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:06.278982 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:06.279030 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 17:06:06.279090 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260808 17:06:06.280253 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:06:06.280491 3170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5606a12af8476db1bf8ec53c7b9a7e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43269 } }
I20260808 17:06:06.280635 3170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:06:06.280673 3170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5606a12af8476db1bf8ec53c7b9a7e, State: Initialized, Role: FOLLOWER
I20260808 17:06:06.280794 3170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5606a12af8476db1bf8ec53c7b9a7e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43269 } }
I20260808 17:06:06.280864 3170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:06:06.280897 3170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:06:06.280942 3170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:06:06.281591 3170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5606a12af8476db1bf8ec53c7b9a7e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43269 } }
I20260808 17:06:06.281695 3170 leader_election.cc:302] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a5606a12af8476db1bf8ec53c7b9a7e; no voters:
I20260808 17:06:06.281858 3170 leader_election.cc:288] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:06:06.281891 3185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:06:06.282053 3185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [term 1 LEADER]: Becoming Leader. State: Replica: 3a5606a12af8476db1bf8ec53c7b9a7e, State: Running, Role: LEADER
I20260808 17:06:06.282111 3170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:06.282150 3185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5606a12af8476db1bf8ec53c7b9a7e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43269 } }
I20260808 17:06:06.282610 3185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a5606a12af8476db1bf8ec53c7b9a7e. Latest consensus state: current_term: 1 leader_uuid: "3a5606a12af8476db1bf8ec53c7b9a7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5606a12af8476db1bf8ec53c7b9a7e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43269 } } }
I20260808 17:06:06.282821 3185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:06.282760 3186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a5606a12af8476db1bf8ec53c7b9a7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5606a12af8476db1bf8ec53c7b9a7e" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43269 } } }
I20260808 17:06:06.283233 3186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:06.283409 3197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:06:06.283946 3197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:06:06.284853 3176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:06.281670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53509 (local address 127.30.194.62:43269)
0808 17:06:06.281706 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:06.281708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:06.281807 (+ 99us) server_negotiation.cc:407] Connection header received
0808 17:06:06.281873 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:06.281875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:06.281887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:06.281946 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:06.282145 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:06.282812 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:06.283740 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:06.284003 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:06.284065 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:06.284625 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:06.284627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:06.284628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:06:06.284632 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:06.284645 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:06:06.284649 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:06.284698 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:06.284766 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 17:06:06.284788 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:06:06.285560 3197 catalog_manager.cc:1347] Generated new cluster ID: 1748d00cf2fd4f9fb383b68f02cf16c1
I20260808 17:06:06.285602 3197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:06:06.293074 3197 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:06:06.293550 3197 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:06:06.298730 3197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a5606a12af8476db1bf8ec53c7b9a7e: Generated new TSK 0
I20260808 17:06:06.298869 3197 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:06:06.301492 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 3035
2026-08-08T17:06:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:06:06 dist-test-slave-2j7r krb5kdc[3216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:06:06 dist-test-slave-2j7r krb5kdc[3216](info): set up 2 sockets
Aug 08 17:06:06 dist-test-slave-2j7r krb5kdc[3216](info): commencing operation
krb5kdc: starting...
W20260808 17:06:07.458849 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:06:07 dist-test-slave-2j7r krb5kdc[3216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:06:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:06:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:06:07.569803 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43595
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:43595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:06:07.647670 3243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:07.647843 3243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:07.647864 3243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:07.648948 3243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:07.648988 3243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:07.649000 3243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:07.649024 3243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:07.649045 3243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:07.650476 3243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:43595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:43595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:07.650688 3243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:07.650873 3243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:07.655324 3255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:07.655335 3261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:07.655314 3257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:07.655362 3243 server_base.cc:1034] running on GCE node
I20260808 17:06:07.655741 3243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:07.656023 3243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:07.657131 3243 hybrid_clock.cc:648] HybridClock initialized: now 1786208767657116 us; error 26 us; skew 500 ppm
Aug 08 17:06:07 dist-test-slave-2j7r krb5kdc[3216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208767, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:07.659370 3243 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:06:07.659770 3243 webserver.cc:460] Webserver started at http://127.30.194.62:40739/ using document root <none> and password file <none>
I20260808 17:06:07.659966 3243 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:07.660012 3243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:07.660127 3243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:07.661041 3243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "18fb215afa914fd18b61e5d1eda71552"
format_stamp: "Formatted at 2026-08-08 17:06:07 on dist-test-slave-2j7r"
server_key: "8116fd5497f1a16a4919e880945e0575"
server_key_iv: "d2ee7db22ce59b1e037980b35f3c90b4"
server_key_version: "encryptionkey@0"
I20260808 17:06:07.661392 3243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "18fb215afa914fd18b61e5d1eda71552"
format_stamp: "Formatted at 2026-08-08 17:06:07 on dist-test-slave-2j7r"
server_key: "8116fd5497f1a16a4919e880945e0575"
server_key_iv: "d2ee7db22ce59b1e037980b35f3c90b4"
server_key_version: "encryptionkey@0"
I20260808 17:06:07.662603 3243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:06:07.663244 3276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:07.663401 3243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:06:07.663491 3243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "18fb215afa914fd18b61e5d1eda71552"
format_stamp: "Formatted at 2026-08-08 17:06:07 on dist-test-slave-2j7r"
server_key: "8116fd5497f1a16a4919e880945e0575"
server_key_iv: "d2ee7db22ce59b1e037980b35f3c90b4"
server_key_version: "encryptionkey@0"
I20260808 17:06:07.663551 3243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:07.666628 3243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:07.668519 3243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:07.668638 3243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:07.672219 3243 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:43595
I20260808 17:06:07.672248 3368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:43595 every 8 connection(s)
I20260808 17:06:07.672516 3243 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:06:07.673408 3369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:07.673657 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 3243
I20260808 17:06:07.673772 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:06:07.673967 31496 external_mini_cluster.cc:1428] Setting key ab3cd77ebddb8b406333c2aabe742f5f
I20260808 17:06:07.675935 3369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552: Bootstrap starting.
I20260808 17:06:07.676077 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:07.674436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48425 (local address 127.30.194.62:43595)
0808 17:06:07.674595 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:07.674602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:07.674625 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:06:07.675000 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:07.675006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:07.675035 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:07.675139 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:07.675678 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:07.675681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:07.675682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:06:07.675685 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:07.675718 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:06:07.675727 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:07.675778 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:07.675859 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 17:06:07.675896 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260808 17:06:07.676591 3369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:07.676890 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:06:07.676890 3369 log.cc:824] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:07.677645 3369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552: No bootstrap required, opened a new log
I20260808 17:06:07.679096 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:07.678311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58665 (local address 127.30.194.62:43595)
0808 17:06:07.678347 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:07.678349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:07.678406 (+ 57us) server_negotiation.cc:407] Connection header received
0808 17:06:07.678469 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:07.678470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:07.678473 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:07.678514 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:07.678935 (+ 421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:07.678936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:07.678936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:06:07.678939 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:07.678949 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:06:07.678952 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:07.678984 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:07.679037 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 17:06:07.679045 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:06:07.679845 3369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fb215afa914fd18b61e5d1eda71552" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43595 } }
I20260808 17:06:07.679965 3369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:06:07.679986 3369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb215afa914fd18b61e5d1eda71552, State: Initialized, Role: FOLLOWER
I20260808 17:06:07.680074 3369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fb215afa914fd18b61e5d1eda71552" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43595 } }
I20260808 17:06:07.680131 3369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:06:07.680155 3369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:06:07.680183 3369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:06:07.680622 3369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fb215afa914fd18b61e5d1eda71552" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43595 } }
I20260808 17:06:07.680701 3369 leader_election.cc:302] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18fb215afa914fd18b61e5d1eda71552; no voters:
I20260808 17:06:07.680811 3369 leader_election.cc:288] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:06:07.680853 3384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:06:07.681010 3369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:07.681023 3384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [term 1 LEADER]: Becoming Leader. State: Replica: 18fb215afa914fd18b61e5d1eda71552, State: Running, Role: LEADER
I20260808 17:06:07.681139 3384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fb215afa914fd18b61e5d1eda71552" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43595 } }
I20260808 17:06:07.681474 3386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18fb215afa914fd18b61e5d1eda71552. Latest consensus state: current_term: 1 leader_uuid: "18fb215afa914fd18b61e5d1eda71552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fb215afa914fd18b61e5d1eda71552" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43595 } } }
I20260808 17:06:07.681560 3386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:07.681708 3385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18fb215afa914fd18b61e5d1eda71552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fb215afa914fd18b61e5d1eda71552" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 43595 } } }
I20260808 17:06:07.681768 3385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552 [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:07.681831 3402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:06:07.682281 3402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:06:07.683727 3402 catalog_manager.cc:1347] Generated new cluster ID: b4aec8b24e8843868f8550cf36b9972d
I20260808 17:06:07.683790 3402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:06:07.694655 3402 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:06:07.694792 3402 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:06:07.699445 3402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18fb215afa914fd18b61e5d1eda71552: Generated new TSK 0
I20260808 17:06:07.699594 3402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:06:07.701960 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 3243
2026-08-08T17:06:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1403 ms)
[----------] 18 tests from AuthTokenIssuingTest (25489 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:06:07 dist-test-slave-2j7r krb5kdc[3430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:06:07 dist-test-slave-2j7r krb5kdc[3430](info): set up 2 sockets
Aug 08 17:06:07 dist-test-slave-2j7r krb5kdc[3430](info): commencing operation
krb5kdc: starting...
W20260808 17:06:08.838073 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:06:08 dist-test-slave-2j7r krb5kdc[3430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:06:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:06:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:06:08.949316 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:45695
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:45695,127.30.194.61:42433,127.30.194.60:37603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:06:09.028420 3450 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:06:09.028621 3450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:09.028644 3450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:09.028669 3450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:09.029810 3450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:09.029850 3450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:09.029870 3450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:09.029886 3450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:09.029901 3450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:09.031389 3450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:45695,127.30.194.61:42433,127.30.194.60:37603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:45695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:09.031625 3450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:09.031816 3450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:06:09.036309 3450 server_base.cc:1034] running on GCE node
W20260808 17:06:09.036237 3460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:09.036346 3462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:09.036238 3459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:09.036700 3450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:09.036962 3450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:09.038095 3450 hybrid_clock.cc:648] HybridClock initialized: now 1786208769038078 us; error 35 us; skew 500 ppm
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:09.044492 3450 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:06:09.044915 3450 webserver.cc:460] Webserver started at http://127.30.194.62:37661/ using document root <none> and password file <none>
I20260808 17:06:09.045141 3450 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:09.045192 3450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:09.045310 3450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:09.046272 3450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "6ddc65a03d3d4c8caca496706d05c2a7"
format_stamp: "Formatted at 2026-08-08 17:06:09 on dist-test-slave-2j7r"
server_key: "92cdc4b682f3a7a5e894087e4e6b9161"
server_key_iv: "a01e16373f2de2cae7b0e8b2f8b0c495"
server_key_version: "encryptionkey@0"
I20260808 17:06:09.046587 3450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "6ddc65a03d3d4c8caca496706d05c2a7"
format_stamp: "Formatted at 2026-08-08 17:06:09 on dist-test-slave-2j7r"
server_key: "92cdc4b682f3a7a5e894087e4e6b9161"
server_key_iv: "a01e16373f2de2cae7b0e8b2f8b0c495"
server_key_version: "encryptionkey@0"
I20260808 17:06:09.047828 3450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:06:09.048491 3480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:09.048640 3450 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:06:09.048722 3450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "6ddc65a03d3d4c8caca496706d05c2a7"
format_stamp: "Formatted at 2026-08-08 17:06:09 on dist-test-slave-2j7r"
server_key: "92cdc4b682f3a7a5e894087e4e6b9161"
server_key_iv: "a01e16373f2de2cae7b0e8b2f8b0c495"
server_key_version: "encryptionkey@0"
I20260808 17:06:09.048781 3450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:09.059931 3450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:09.061861 3450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:09.061992 3450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:09.065620 3450 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:45695
I20260808 17:06:09.065657 3567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:45695 every 8 connection(s)
I20260808 17:06:09.065968 3450 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:06:09.066933 3568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:09.068328 3568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:09.073280 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 3450
I20260808 17:06:09.073374 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:06:09.073575 31496 external_mini_cluster.cc:1428] Setting key b8e7ee9ca8d98d8fc2be22546441bb4b
I20260808 17:06:09.076884 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.068890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:45695 (local address 127.0.0.1:51664)
0808 17:06:09.069034 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.069064 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:09.069528 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:09.069753 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:09.069756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:09.069982 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:09.070124 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.070128 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.071020 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.071021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.071662 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.071664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.072077 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.072078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.072125 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.072475 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:09.072483 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:09.073855 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:09.075511 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.075519 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.075521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.075767 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.075974 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.075975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.075975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.076078 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.076363 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:09.076369 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:09.076442 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 17:06:09.076535 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 17:06:09.076690 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260808 17:06:09.076901 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.069044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51664 (local address 127.30.194.62:45695)
0808 17:06:09.069160 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.069164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.069195 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:06:09.069644 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.069645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.069660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.069730 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.070192 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.070952 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.071724 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.072007 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.072073 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.073932 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.073942 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.073942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.073948 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.075434 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.075838 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.075840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.075841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.075922 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.076133 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.076134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.076134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.076304 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.076359 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.076575 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.076757 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
I20260808 17:06:09.077581 3568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:09.077876 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.077772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42433 (local address 127.0.0.1:44358)
0808 17:06:09.077814 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.077856 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:42433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 17:06:09.078001 3488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.194.61:42433: connect: Connection refused (error 111)
I20260808 17:06:09.078352 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.078270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42433 (local address 127.0.0.1:44368)
0808 17:06:09.078307 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.078331 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:42433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 17:06:09.078462 3568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.61:42433: Network error: Client connection negotiation failed: client connection to 127.30.194.61:42433: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.30.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab.
Entry for principal kudu/127.30.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab.
I20260808 17:06:09.095500 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.61:42433
--webserver_interface=127.30.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab
--principal=kudu/127.30.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:45695,127.30.194.61:42433,127.30.194.60:37603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:06:09.127672 3568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } attempt: 1
I20260808 17:06:09.128152 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.128021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42433 (local address 127.0.0.1:44378)
0808 17:06:09.128082 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.128123 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:42433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 17:06:09.128526 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.128427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42433 (local address 127.0.0.1:44388)
0808 17:06:09.128475 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.128503 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:42433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 17:06:09.128625 3568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.61:42433: Network error: Client connection negotiation failed: client connection to 127.30.194.61:42433: connect: Connection refused (error 111)
W20260808 17:06:09.174049 3585 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:06:09.174247 3585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:09.174270 3585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:09.174295 3585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:09.175411 3585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:09.175453 3585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:09.175472 3585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:09.175489 3585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:09.175503 3585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:09.176920 3585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:45695,127.30.194.61:42433,127.30.194.60:37603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab
--principal=kudu/127.30.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.61:42433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:09.177174 3585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:09.177364 3585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:09.181849 3595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:09.181900 3591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:09.181907 3585 server_base.cc:1034] running on GCE node
W20260808 17:06:09.181829 3592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:09.182300 3585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:09.182547 3585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:09.183693 3585 hybrid_clock.cc:648] HybridClock initialized: now 1786208769183677 us; error 27 us; skew 500 ppm
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:09.186062 3585 init.cc:377] Logged in from keytab as kudu/127.30.194.61@KRBTEST.COM (short username kudu)
I20260808 17:06:09.186450 3585 webserver.cc:460] Webserver started at http://127.30.194.61:44283/ using document root <none> and password file <none>
I20260808 17:06:09.186635 3585 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:09.186681 3585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:09.186815 3585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:09.187880 3585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/data/instance:
uuid: "3220bbb8e93a4264b4490cd3e85eebc1"
format_stamp: "Formatted at 2026-08-08 17:06:09 on dist-test-slave-2j7r"
server_key: "21dbd7cc144ce5cc97da7f1ebd46f29d"
server_key_iv: "b12df37892f8c9c521e58fca6be642b6"
server_key_version: "encryptionkey@0"
I20260808 17:06:09.188200 3585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/wal/instance:
uuid: "3220bbb8e93a4264b4490cd3e85eebc1"
format_stamp: "Formatted at 2026-08-08 17:06:09 on dist-test-slave-2j7r"
server_key: "21dbd7cc144ce5cc97da7f1ebd46f29d"
server_key_iv: "b12df37892f8c9c521e58fca6be642b6"
server_key_version: "encryptionkey@0"
I20260808 17:06:09.189443 3585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:06:09.190109 3607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:09.190249 3585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:06:09.190331 3585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/wal
uuid: "3220bbb8e93a4264b4490cd3e85eebc1"
format_stamp: "Formatted at 2026-08-08 17:06:09 on dist-test-slave-2j7r"
server_key: "21dbd7cc144ce5cc97da7f1ebd46f29d"
server_key_iv: "b12df37892f8c9c521e58fca6be642b6"
server_key_version: "encryptionkey@0"
I20260808 17:06:09.190388 3585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:09.196765 3568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } attempt: 2
I20260808 17:06:09.197149 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.197004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42433 (local address 127.0.0.1:44398)
0808 17:06:09.197060 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.197112 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:42433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 17:06:09.197466 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.197388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42433 (local address 127.0.0.1:44400)
0808 17:06:09.197426 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.197450 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:42433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 17:06:09.197551 3568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.61:42433: Network error: Client connection negotiation failed: client connection to 127.30.194.61:42433: connect: Connection refused (error 111)
I20260808 17:06:09.228787 3585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:09.231037 3585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:09.231177 3585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:09.234939 3585 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.61:42433
I20260808 17:06:09.234987 3708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.61:42433 every 8 connection(s)
I20260808 17:06:09.235316 3585 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
I20260808 17:06:09.236256 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:09.237818 3709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:09.239804 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 3585
I20260808 17:06:09.239924 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-1/wal/instance
I20260808 17:06:09.240111 31496 external_mini_cluster.cc:1428] Setting key 0bf1fde63e66cfe6bdf05534976cd8b7
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:06:09.246788 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.238463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51666 (local address 127.30.194.62:45695)
0808 17:06:09.238514 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.238516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.238744 (+ 228us) server_negotiation.cc:407] Connection header received
0808 17:06:09.239173 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.239174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.239191 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.239263 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.239958 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.240498 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.241796 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.242105 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.242176 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.244145 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.244147 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.244147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.244154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.245535 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.245912 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.245913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.245913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.245981 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.246192 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.246193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.246194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.246302 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.246357 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.246607 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.246687 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:06:09.246928 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.238425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:45695 (local address 127.0.0.1:51666)
0808 17:06:09.238589 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.238630 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:09.239065 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:09.239321 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:09.239324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:09.239566 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:09.239840 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.239846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.240590 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.240592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.241707 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.241712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.242194 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.242195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.242290 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.242720 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:09.242736 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:09.244058 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:09.245613 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.245621 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.245623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.245842 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.246036 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.246037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.246037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.246135 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.246357 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:09.246364 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:09.246492 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 17:06:09.246578 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 17:06:09.246720 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260808 17:06:09.247463 3709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.194.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.60@KRBTEST.COM" created.
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
I20260808 17:06:09.254060 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.247687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42433 (local address 127.0.0.1:44414)
0808 17:06:09.247736 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.247748 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:09.247793 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:09.248006 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:09.248008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:09.248142 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:09.248198 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.248200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.249071 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.249072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.249571 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.249573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.249991 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.249992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.250038 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.250342 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:09.250345 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:09.251515 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:09.253102 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.253103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.253104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.253249 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.253423 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.253424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.253425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.253479 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.253754 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:09.253755 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:09.253801 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 17:06:09.253872 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:06:09.253965 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 17:06:09.254215 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.247743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.30.194.61:42433)
0808 17:06:09.247883 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.247887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.247906 (+ 19us) server_negotiation.cc:407] Connection header received
0808 17:06:09.247924 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.247926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.247936 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.247993 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.248271 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.248957 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.249632 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.249936 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.249991 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.251597 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.251606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.251606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.251613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.253012 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.253315 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.253318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.253318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.253370 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.253545 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.253546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.253546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.253685 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.253759 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.253898 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.254021 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260808 17:06:09.254752 3709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:09.255061 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.254948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53822)
0808 17:06:09.254991 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.255040 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 17:06:09.255191 3613 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
I20260808 17:06:09.255574 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.255494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53832)
0808 17:06:09.255532 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.255556 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 17:06:09.255683 3709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:37603: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab.
Entry for principal kudu/127.30.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab.
I20260808 17:06:09.264816 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.60:37603
--webserver_interface=127.30.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab
--principal=kudu/127.30.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:45695,127.30.194.61:42433,127.30.194.60:37603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:06:09.288694 3568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } attempt: 3
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
I20260808 17:06:09.295557 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.289159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.30.194.61:42433)
0808 17:06:09.289224 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.289226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.289238 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:06:09.289294 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.289295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.289310 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.289366 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.289702 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.290194 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.290887 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.291124 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.291185 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.293129 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.293131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.293131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.293140 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.294507 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.294887 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.294888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.294888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.294948 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.295126 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.295127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.295127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.295230 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.295277 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.295430 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.295489 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:06:09.295608 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.289064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42433 (local address 127.0.0.1:44424)
0808 17:06:09.289124 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.289136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:09.289223 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:09.289378 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:09.289379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:09.289544 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:09.289614 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.289616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.290277 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.290278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.290795 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.290798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.291201 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.291202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.291278 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.291729 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:09.291733 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:09.292998 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:09.294589 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.294590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.294591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.294798 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.295011 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.295012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.295012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.295075 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.295282 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:09.295283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:09.295333 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 17:06:09.295414 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:06:09.295509 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 17:06:09.296005 3568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:09.296347 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.296249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53844)
0808 17:06:09.296292 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.296324 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 17:06:09.296626 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.296550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53850)
0808 17:06:09.296585 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.296609 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 17:06:09.296723 3568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:37603: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
I20260808 17:06:09.304908 3709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } attempt: 1
I20260808 17:06:09.305307 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.305195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53852)
0808 17:06:09.305245 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.305279 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 17:06:09.305625 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.305543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53856)
0808 17:06:09.305582 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.305607 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 17:06:09.305724 3709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:37603: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
I20260808 17:06:09.327862 3568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } attempt: 1
I20260808 17:06:09.328264 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.328122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53858)
0808 17:06:09.328179 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.328225 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 17:06:09.328622 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.328518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53862)
0808 17:06:09.328575 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.328604 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 17:06:09.328719 3568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:37603: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
W20260808 17:06:09.347010 3722 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:06:09.347193 3722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:09.347208 3722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:09.347225 3722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:09.348277 3722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:09.348307 3722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:09.348320 3722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:09.348331 3722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:09.348341 3722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:09.349777 3722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:45695,127.30.194.61:42433,127.30.194.60:37603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab
--principal=kudu/127.30.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.60:37603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:09.349968 3722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:09.350134 3722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:09.354527 3738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:09.354538 3733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:09.354542 3734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:09.354737 3722 server_base.cc:1034] running on GCE node
I20260808 17:06:09.354939 3722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:09.355157 3722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:09.356297 3722 hybrid_clock.cc:648] HybridClock initialized: now 1786208769356285 us; error 29 us; skew 500 ppm
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:09.358610 3722 init.cc:377] Logged in from keytab as kudu/127.30.194.60@KRBTEST.COM (short username kudu)
I20260808 17:06:09.359026 3722 webserver.cc:460] Webserver started at http://127.30.194.60:42071/ using document root <none> and password file <none>
I20260808 17:06:09.359212 3722 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:09.359258 3722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:09.359385 3722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:09.360348 3722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/data/instance:
uuid: "03d6f4d30f524b0184c28b1b56d4d9a9"
format_stamp: "Formatted at 2026-08-08 17:06:09 on dist-test-slave-2j7r"
server_key: "57e04ac876b7c11735891f9f9ef4f95d"
server_key_iv: "93a2de205fdec652ced835ea81680ace"
server_key_version: "encryptionkey@0"
I20260808 17:06:09.360669 3722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/wal/instance:
uuid: "03d6f4d30f524b0184c28b1b56d4d9a9"
format_stamp: "Formatted at 2026-08-08 17:06:09 on dist-test-slave-2j7r"
server_key: "57e04ac876b7c11735891f9f9ef4f95d"
server_key_iv: "93a2de205fdec652ced835ea81680ace"
server_key_version: "encryptionkey@0"
I20260808 17:06:09.361927 3722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:06:09.362542 3754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:09.362692 3722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:06:09.362773 3722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/wal
uuid: "03d6f4d30f524b0184c28b1b56d4d9a9"
format_stamp: "Formatted at 2026-08-08 17:06:09 on dist-test-slave-2j7r"
server_key: "57e04ac876b7c11735891f9f9ef4f95d"
server_key_iv: "93a2de205fdec652ced835ea81680ace"
server_key_version: "encryptionkey@0"
I20260808 17:06:09.362828 3722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:09.373867 3709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } attempt: 2
I20260808 17:06:09.374235 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.374102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53874)
0808 17:06:09.374163 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.374200 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:06:09.374606 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.374506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53884)
0808 17:06:09.374554 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.374583 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 17:06:09.374774 3709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:37603: Network error: Client connection negotiation failed: client connection to 127.30.194.60:37603: connect: Connection refused (error 111)
I20260808 17:06:09.375567 3722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:09.377238 3722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:09.377363 3722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:09.380923 3722 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.60:37603
I20260808 17:06:09.380951 3855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.60:37603 every 8 connection(s)
I20260808 17:06:09.381302 3722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
I20260808 17:06:09.382180 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:09.383548 3858 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:06:09.389356 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 3722
I20260808 17:06:09.389479 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208501350379-31496-0/minicluster-data/master-2/wal/instance
I20260808 17:06:09.389678 31496 external_mini_cluster.cc:1428] Setting key 7dca60e25c9deb3d1fa335b5b4ded377
I20260808 17:06:09.391606 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.384149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51682 (local address 127.30.194.62:45695)
0808 17:06:09.384202 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.384204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.384385 (+ 181us) server_negotiation.cc:407] Connection header received
0808 17:06:09.384855 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.384856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.384870 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.384929 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.385544 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.386018 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.386970 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.387154 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.387209 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.389062 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.389063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.389064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.389070 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.390405 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.390753 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.390754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.390755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.390824 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.391029 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.391030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.391031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.391138 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.391193 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.391437 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.391518 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:06:09.391783 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.384121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:45695 (local address 127.0.0.1:51682)
0808 17:06:09.384279 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.384312 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:09.384762 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:09.384999 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:09.385003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:09.385277 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:09.385461 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.385465 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.386095 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.386096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.386895 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.386897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.387238 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.387240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.387345 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.387719 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:09.387729 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:09.388981 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:09.390476 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.390483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.390485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.390688 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.390877 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.390878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.390878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.390974 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.391190 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:09.391197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:09.391319 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 17:06:09.391411 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 17:06:09.391586 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":24,"thread_start_us":79,"threads_started":1}
I20260808 17:06:09.392318 3858 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
I20260808 17:06:09.398011 3861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.390014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53894 (local address 127.30.194.60:37603)
0808 17:06:09.390152 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.390156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.390297 (+ 141us) server_negotiation.cc:407] Connection header received
0808 17:06:09.390347 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.390349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.390361 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.390423 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.390958 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.391601 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.392471 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.392874 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.392936 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.395251 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.395257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.395258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.395263 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.396777 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.397167 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.397170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.397170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.397224 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.397407 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.397408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.397408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.397519 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.397669 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.397749 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.397820 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260808 17:06:09.398375 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.392529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:42433 (local address 127.0.0.1:44428)
0808 17:06:09.392638 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.392653 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:09.392709 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:09.392822 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:09.392823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:09.392942 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:09.392983 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.392984 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.393718 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.393719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.394180 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.394181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.394521 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.394522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.394593 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.394924 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:09.394929 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:09.395987 (+ 1058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:09.397528 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.397529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.397529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.397684 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.397850 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.397851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.397851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.397906 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.398144 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:09.398145 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:09.398184 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 17:06:09.398248 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:06:09.398321 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 17:06:09.398454 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.392598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.30.194.61:42433)
0808 17:06:09.392635 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.392637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.392696 (+ 59us) server_negotiation.cc:407] Connection header received
0808 17:06:09.392767 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.392767 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.392777 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.392821 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.393046 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.393632 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.394254 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.394457 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.394508 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.396059 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.396060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.396061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.396067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.397466 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.397745 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.397746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.397746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.397799 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.397954 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.397955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.397955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.398058 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.398116 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.398286 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.398363 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:06:09.398707 3858 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:06:09.403848 3568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } attempt: 2
I20260808 17:06:09.404419 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.398958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53904)
0808 17:06:09.399092 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.399099 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:09.399163 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:09.399275 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:09.399276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:09.399373 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:09.399397 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.399398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.399981 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.399982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.400444 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.400445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.400794 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.400795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.400839 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.401179 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:09.401184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:09.402220 (+ 1036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:09.403516 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.403517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.403517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.403658 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.403825 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.403826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.403826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.403876 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.404136 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:09.404137 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:09.404171 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 17:06:09.404230 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:06:09.404319 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260808 17:06:09.404490 3861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.399064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53904 (local address 127.30.194.60:37603)
0808 17:06:09.399110 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.399111 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.399157 (+ 46us) server_negotiation.cc:407] Connection header received
0808 17:06:09.399197 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.399198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.399207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.399265 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.399454 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.399910 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.400510 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.400735 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.400786 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.402294 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.402296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.402296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.402303 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.403449 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.403712 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.403712 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.403713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.403757 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.403924 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.403924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.403924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.404061 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.404140 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.404255 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.404376 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":28}
I20260808 17:06:09.405493 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.398882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51692 (local address 127.30.194.62:45695)
0808 17:06:09.398918 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.398920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.398928 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:06:09.398970 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.398970 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.398979 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.399026 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.399517 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.399948 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.400564 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.400743 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.400795 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.402983 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.402984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.402984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.402991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.404347 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.404711 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.404712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.404713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.404848 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.405026 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.405027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.405027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.405163 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.405315 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.405351 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.405413 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:06:09.406174 3858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9: Bootstrap starting.
I20260808 17:06:09.406848 3858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:09.407159 3858 log.cc:824] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:09.407917 3858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9: No bootstrap required, opened a new log
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
I20260808 17:06:09.409881 3858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } }
I20260808 17:06:09.410020 3858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03d6f4d30f524b0184c28b1b56d4d9a9, State: Initialized, Role: FOLLOWER
I20260808 17:06:09.410097 3858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } }
I20260808 17:06:09.410362 3870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } } }
I20260808 17:06:09.410422 3858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:09.410524 3870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:09.411177 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.404063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53918)
0808 17:06:09.404105 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.404114 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:09.404146 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:09.404473 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:09.404474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:09.404618 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:09.404656 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.404657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.405801 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.405802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.406508 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.406511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.406991 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.406992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.407047 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.407406 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:09.407412 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:09.408645 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:09.410080 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.410081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.410081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.410291 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.410501 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.410502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.410502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.410562 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.410863 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:09.410864 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:09.410920 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 17:06:09.410994 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 17:06:09.411086 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 17:06:09.411478 3866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.404212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53918 (local address 127.30.194.60:37603)
0808 17:06:09.404341 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.404343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.404349 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:06:09.404364 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.404364 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.404373 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.404897 (+ 524us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.404936 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.405707 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.406640 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.406923 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.406989 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.408734 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.408735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.408736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.408742 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.409987 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.410374 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.410375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.410375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.410429 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.410632 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.410633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.410633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.410771 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.410859 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.411307 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.411377 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260808 17:06:09.413133 3568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7: Bootstrap starting.
I20260808 17:06:09.413496 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.406001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44432 (local address 127.30.194.61:42433)
0808 17:06:09.406031 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.406033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.406040 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:06:09.406069 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.406069 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.406077 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.406117 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.406510 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.407000 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.407728 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.407997 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.408064 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.410476 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.410478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.410478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.410484 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.412063 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.412446 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.412447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.412448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.412506 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.412713 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.412714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.412715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.412828 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.412982 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.413347 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.413417 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 17:06:09.412248 3890 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 17:06:09.413779 3568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7: Neither blocks nor log segments found. Creating new log.
W20260808 17:06:09.413793 3890 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:06:09.414062 3568 log.cc:824] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:09.415007 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:06:09.415036 3568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7: No bootstrap required, opened a new log
I20260808 17:06:09.415974 3524 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6ddc65a03d3d4c8caca496706d05c2a7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51692
I20260808 17:06:09.414703 3722 master_runner.cc:186] Error getting master registration for 127.30.194.62:45695: OK, instance_id { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" instance_seqno: 1786208769064590 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:06:09.416824 3722 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:06:09.416915 3568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } }
I20260808 17:06:09.417038 3568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddc65a03d3d4c8caca496706d05c2a7, State: Initialized, Role: FOLLOWER
I20260808 17:06:09.417119 3568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } }
I20260808 17:06:09.417418 3568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:09.417482 3524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 17:06:09.417536 3524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:06:09.418354 3524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } }
I20260808 17:06:09.418591 3524 leader_election.cc:288] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [CANDIDATE]: Term 1 election: Requested vote from peers 3220bbb8e93a4264b4490cd3e85eebc1 (127.30.194.61:42433), 03d6f4d30f524b0184c28b1b56d4d9a9 (127.30.194.60:37603)
I20260808 17:06:09.417373 3894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } } }
I20260808 17:06:09.418715 3894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:09.419313 3803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9"
I20260808 17:06:09.419422 3803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:06:09.420420 3803 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddc65a03d3d4c8caca496706d05c2a7 in term 1.
I20260808 17:06:09.420648 3488 leader_election.cc:302] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03d6f4d30f524b0184c28b1b56d4d9a9, 6ddc65a03d3d4c8caca496706d05c2a7; no voters:
W20260808 17:06:09.421988 3913 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:06:09.422046 3913 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:06:09.419792 3450 master_runner.cc:186] Error getting master registration for 127.30.194.61:42433: OK, instance_id { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" instance_seqno: 1786208769233920 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:06:09.422518 3450 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:06:09.422753 3894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:06:09.422911 3894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [term 1 LEADER]: Becoming Leader. State: Replica: 6ddc65a03d3d4c8caca496706d05c2a7, State: Running, Role: LEADER
I20260808 17:06:09.423002 3894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } }
I20260808 17:06:09.424045 3894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ddc65a03d3d4c8caca496706d05c2a7. Latest consensus state: current_term: 1 leader_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } } }
I20260808 17:06:09.424132 3894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:09.431867 3932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:06:09.432391 3932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:06:09.433895 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.422587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51694 (local address 127.30.194.62:45695)
0808 17:06:09.422621 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.422623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.422630 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:06:09.422647 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.422648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.422656 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.422705 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.424280 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.424930 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.426002 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.426291 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.426334 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.428746 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.428747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.428747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.428754 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.430480 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.433016 (+ 2536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.433017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.433017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.433076 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.433329 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.433330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.433330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.433450 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.433520 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.433749 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.433814 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:06:09.435504 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.421279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44446 (local address 127.30.194.61:42433)
0808 17:06:09.421318 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.421319 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.421640 (+ 321us) server_negotiation.cc:407] Connection header received
0808 17:06:09.421755 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.421767 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.421777 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.421803 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.423292 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.424009 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.425603 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.426375 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.426421 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.428701 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.428702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.428703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.428709 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.431617 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.432603 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.432604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.432604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.432676 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.434761 (+ 2085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.434762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.434762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.434906 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.434985 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.435300 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.435412 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:06:09.435636 3866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.421609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53926 (local address 127.30.194.60:37603)
0808 17:06:09.421643 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.421644 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.422188 (+ 544us) server_negotiation.cc:407] Connection header received
0808 17:06:09.422388 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.422389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.422398 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.422424 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.429161 (+ 6737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.429915 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.430847 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.431171 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.431230 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.432560 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.432561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.432561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.432568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.434254 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.434615 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.434616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.434616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.434687 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.434894 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.434894 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.434895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.435021 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.435116 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.435466 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.435556 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:06:09.441753 3803 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [term 1 FOLLOWER]: Refusing update from remote peer 6ddc65a03d3d4c8caca496706d05c2a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:06:09.442101 3930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:06:09.443396 3870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ddc65a03d3d4c8caca496706d05c2a7. Latest consensus state: current_term: 1 leader_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } } }
I20260808 17:06:09.443481 3870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:09.445231 3894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } } }
I20260808 17:06:09.445312 3894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:09.445725 3870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } } }
I20260808 17:06:09.445796 3870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:09.445793 3932 catalog_manager.cc:1347] Generated new cluster ID: e3ee5ce14e594e85ab6daab6dc34dbdc
I20260808 17:06:09.445835 3932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:06:09.446209 3937 mvcc.cc:205] Tried to move back new op lower bound from 7316311119631228928 to 7316311119559958528. Current Snapshot: MvccSnapshot[applied={T|T < 7316311119631228928 or (T in {7316311119631228928})}]
I20260808 17:06:09.446357 3939 mvcc.cc:205] Tried to move back new op lower bound from 7316311119631228928 to 7316311119559958528. Current Snapshot: MvccSnapshot[applied={T|T < 7316311119631228928}]
I20260808 17:06:09.457746 3932 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:06:09.458951 3932 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:06:09.465901 3709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } attempt: 3
Aug 08 17:06:09 dist-test-slave-2j7r krb5kdc[3430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
I20260808 17:06:09.473907 3932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7: Generated new TSK 0
I20260808 17:06:09.474236 3932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:06:09.475167 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.466222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:37603 (local address 127.0.0.1:53938)
0808 17:06:09.466264 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:09.466277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:09.466335 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:09.466529 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:09.466530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:09.466713 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:09.466759 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.466761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.467672 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.467673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.468488 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:09.468490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.469719 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.469720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:09.469780 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.470232 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:09.470238 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:09.471749 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:09.473887 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.473888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.473889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.474098 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.474422 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:09.474423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:09.474423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:09.474486 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:09.474765 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:09.474765 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:09.474911 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 17:06:09.474977 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 17:06:09.475065 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 17:06:09.475183 3866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:09.466239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53938 (local address 127.30.194.60:37603)
0808 17:06:09.466279 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.466281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.466322 (+ 41us) server_negotiation.cc:407] Connection header received
0808 17:06:09.466392 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.466393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.466403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.466458 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.466859 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.467584 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.469315 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:09.469609 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:09.469666 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:09.471837 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:09.471838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:09.471839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:09.471846 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:09.473795 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.474249 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.474250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.474251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.474311 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:09.474557 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:09.474558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:09.474558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:09.474692 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:09.474770 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:09.475017 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 17:06:09.475092 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:06:09.477111 3709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1: Bootstrap starting.
I20260808 17:06:09.477823 3709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:09.478096 3709 log.cc:824] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:09.479467 3709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1: No bootstrap required, opened a new log
I20260808 17:06:09.483321 3709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } }
I20260808 17:06:09.483505 3709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3220bbb8e93a4264b4490cd3e85eebc1, State: Initialized, Role: FOLLOWER
I20260808 17:06:09.483610 3709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } }
I20260808 17:06:09.483955 3709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:09.484027 3665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:06:09.486244 3665 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [term 1 FOLLOWER]: Refusing update from remote peer 6ddc65a03d3d4c8caca496706d05c2a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:06:09.486536 3930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:06:09.484050 3667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3220bbb8e93a4264b4490cd3e85eebc1"
I20260808 17:06:09.487763 3667 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddc65a03d3d4c8caca496706d05c2a7 in term 1.
I20260808 17:06:09.490020 3983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ddc65a03d3d4c8caca496706d05c2a7. Latest consensus state: current_term: 1 leader_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } } }
I20260808 17:06:09.483932 3950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } } }
I20260808 17:06:09.490137 3950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:09.490207 3950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } } }
I20260808 17:06:09.490268 3950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 17:06:09.490984 3984 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:06:09.491078 3984 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:06:09.491210 3983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:09.497713 3977 mvcc.cc:205] Tried to move back new op lower bound from 7316311119757606912 to 7316311119559958528. Current Snapshot: MvccSnapshot[applied={T|T < 7316311119631228928}]
W20260808 17:06:09.498957 3577 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.497658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51698 (local address 127.30.194.62:45695)
0808 17:06:09.497718 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.497720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.498164 (+ 444us) server_negotiation.cc:407] Connection header received
0808 17:06:09.498359 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.498360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.498372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.498404 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.498891 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51698: BlockingRecv error: recv got EOF from 127.0.0.1:51698 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 17:06:09.502353 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.501403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44450 (local address 127.30.194.61:42433)
0808 17:06:09.501445 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.501447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.501829 (+ 382us) server_negotiation.cc:407] Connection header received
0808 17:06:09.501929 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.501930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.501940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.501972 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.502297 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44450: BlockingRecv error: recv got EOF from 127.0.0.1:44450 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 17:06:09.503378 3866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.501708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53952 (local address 127.30.194.60:37603)
0808 17:06:09.501750 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.501752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.502804 (+ 1052us) server_negotiation.cc:407] Connection header received
0808 17:06:09.502939 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.502940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.502952 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.502985 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.503315 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53952: BlockingRecv error: recv got EOF from 127.0.0.1:53952 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 17:06:09.505961 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.504703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44464 (local address 127.30.194.61:42433)
0808 17:06:09.504751 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.504753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.505383 (+ 630us) server_negotiation.cc:407] Connection header received
0808 17:06:09.505517 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.505519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.505538 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.505570 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.505897 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44464: BlockingRecv error: recv got EOF from 127.0.0.1:44464 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 17:06:09.509128 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.508264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.30.194.61:42433)
0808 17:06:09.508298 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.508300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.508559 (+ 259us) server_negotiation.cc:407] Connection header received
0808 17:06:09.508659 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.508660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.508671 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.508716 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.509051 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44480: BlockingRecv error: recv got EOF from 127.0.0.1:44480 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 17:06:09.513238 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.512355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44486 (local address 127.30.194.61:42433)
0808 17:06:09.512390 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.512392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.512674 (+ 282us) server_negotiation.cc:407] Connection header received
0808 17:06:09.512797 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.512798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.512809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.512841 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.513199 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44486: BlockingRecv error: recv got EOF from 127.0.0.1:44486 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 17:06:09.519232 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.518399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.30.194.61:42433)
0808 17:06:09.518434 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.518436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.518673 (+ 237us) server_negotiation.cc:407] Connection header received
0808 17:06:09.518818 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.518819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.518830 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.518861 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.519192 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44490: BlockingRecv error: recv got EOF from 127.0.0.1:44490 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 17:06:09.529340 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.528427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44500 (local address 127.30.194.61:42433)
0808 17:06:09.528464 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.528466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.528739 (+ 273us) server_negotiation.cc:407] Connection header received
0808 17:06:09.528846 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.528847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.528857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.528888 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.529299 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44500: BlockingRecv error: recv got EOF from 127.0.0.1:44500 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 17:06:09.547322 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.546500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44506 (local address 127.30.194.61:42433)
0808 17:06:09.546536 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.546538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.546806 (+ 268us) server_negotiation.cc:407] Connection header received
0808 17:06:09.546899 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.546900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.546910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.546937 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.547258 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44506: BlockingRecv error: recv got EOF from 127.0.0.1:44506 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 17:06:09.581398 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.580664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44516 (local address 127.30.194.61:42433)
0808 17:06:09.580708 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.580710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.580866 (+ 156us) server_negotiation.cc:407] Connection header received
0808 17:06:09.580918 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.580919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.580931 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.580977 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.581339 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44516: BlockingRecv error: recv got EOF from 127.0.0.1:44516 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 17:06:09.646981 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.646381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44532 (local address 127.30.194.61:42433)
0808 17:06:09.646442 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.646444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.646523 (+ 79us) server_negotiation.cc:407] Connection header received
0808 17:06:09.646595 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.646596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.646606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.646650 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.646934 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44532: BlockingRecv error: recv got EOF from 127.0.0.1:44532 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 17:06:09.776798 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:09.776094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44538 (local address 127.30.194.61:42433)
0808 17:06:09.776169 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:09.776172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:09.776263 (+ 91us) server_negotiation.cc:407] Connection header received
0808 17:06:09.776345 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:09.776346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:09.776358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:09.776402 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:09.776735 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44538: BlockingRecv error: recv got EOF from 127.0.0.1:44538 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 17:06:09.946872 3930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ddc65a03d3d4c8caca496706d05c2a7" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 45695 } } peers { permanent_uuid: "3220bbb8e93a4264b4490cd3e85eebc1" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 42433 } } peers { permanent_uuid: "03d6f4d30f524b0184c28b1b56d4d9a9" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 37603 } } }
I20260808 17:06:09.947027 3930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ddc65a03d3d4c8caca496706d05c2a7 [sys.catalog]: This master's current role is: LEADER
W20260808 17:06:10.034544 3718 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:10.033964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.30.194.61:42433)
0808 17:06:10.034013 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:10.034015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:10.034073 (+ 58us) server_negotiation.cc:407] Connection header received
0808 17:06:10.034146 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:10.034146 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:10.034156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:10.034192 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:10.034483 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44542: BlockingRecv error: recv got EOF from 127.0.0.1:44542 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:06:10.414198 3890 catalog_manager.cc:1259] Loaded cluster ID: e3ee5ce14e594e85ab6daab6dc34dbdc
I20260808 17:06:10.414263 3890 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9: loading cluster ID for follower catalog manager: success
I20260808 17:06:10.415040 3890 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9: acquiring CA information for follower catalog manager: success
I20260808 17:06:10.415289 3890 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 03d6f4d30f524b0184c28b1b56d4d9a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:06:10.491533 3984 catalog_manager.cc:1259] Loaded cluster ID: e3ee5ce14e594e85ab6daab6dc34dbdc
I20260808 17:06:10.491608 3984 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1: loading cluster ID for follower catalog manager: success
I20260808 17:06:10.492506 3984 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1: acquiring CA information for follower catalog manager: success
I20260808 17:06:10.492825 3984 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3220bbb8e93a4264b4490cd3e85eebc1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:06:10.549824 4101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:10.547621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.30.194.61:42433)
0808 17:06:10.547737 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:10.547739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:10.547765 (+ 26us) server_negotiation.cc:407] Connection header received
0808 17:06:10.547830 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:10.547831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:10.547873 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:10.547944 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:06:10.548051 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:10.548541 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:10.549235 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:10.549412 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:10.549452 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:10.549556 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:06:10.549658 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:06:10.549708 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:10.549729 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 17:06:10.549745 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260808 17:06:10.553717 4114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:10.551474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53958 (local address 127.30.194.60:37603)
0808 17:06:10.551567 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:10.551570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:10.551612 (+ 42us) server_negotiation.cc:407] Connection header received
0808 17:06:10.551675 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:10.551675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:10.551707 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:10.551770 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:06:10.551836 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:10.552451 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:10.553115 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:10.553347 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:10.553398 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:10.553482 (+ 84us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:06:10.553569 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:06:10.553614 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:10.553629 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 17:06:10.553643 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":43,"threads_started":1}
I20260808 17:06:10.557664 4114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:10.555380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.30.194.60:37603)
0808 17:06:10.555412 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:10.555414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:10.555463 (+ 49us) server_negotiation.cc:407] Connection header received
0808 17:06:10.555525 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:10.555526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:10.555535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:10.555577 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:06:10.555687 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:10.556121 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:10.556987 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:10.557300 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:10.557358 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:10.557471 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:06:10.557499 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:06:10.557537 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:10.557586 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 17:06:10.557603 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:06:10.558136 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:10.555195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.30.194.62:45695)
0808 17:06:10.555295 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:10.555297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:10.555305 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:06:10.555409 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:10.555410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:10.555462 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:10.555539 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:06:10.555732 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:10.556478 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:10.557171 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:10.557398 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:10.557454 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:10.557659 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:06:10.557841 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:06:10.557889 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:10.558033 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 17:06:10.558057 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":53,"threads_started":1}
I20260808 17:06:10.558212 4101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:10.555199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.30.194.61:42433)
0808 17:06:10.555241 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:10.555243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:10.555419 (+ 176us) server_negotiation.cc:407] Connection header received
0808 17:06:10.555509 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:10.555509 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:10.555518 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:10.555543 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:06:10.555702 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:10.556407 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:10.557440 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:10.557706 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:10.557761 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:10.557945 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:06:10.557974 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:06:10.558024 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:10.558134 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 17:06:10.558154 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:06:10.559551 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 3450
I20260808 17:06:10.570449 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 3585
I20260808 17:06:10.578850 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 3722
2026-08-08T17:06:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2871 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:06:10 dist-test-slave-2j7r krb5kdc[4134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:06:10 dist-test-slave-2j7r krb5kdc[4134](info): set up 2 sockets
Aug 08 17:06:10 dist-test-slave-2j7r krb5kdc[4134](info): commencing operation
krb5kdc: starting...
W20260808 17:06:11.733058 31496 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/test-user.keytab.
Aug 08 17:06:11 dist-test-slave-2j7r krb5kdc[4134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:06:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:06:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
Entry for principal kudu/127.30.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab.
I20260808 17:06:11.842096 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40601
--webserver_interface=127.30.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:40601,127.30.194.61:46431,127.30.194.60:41603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:06:11.921974 4161 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:06:11.922156 4161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:11.922179 4161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:11.922217 4161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:11.923317 4161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:11.923354 4161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:11.923373 4161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:11.923388 4161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:11.923404 4161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:11.924820 4161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:40601,127.30.194.61:46431,127.30.194.60:41603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.62.keytab
--principal=kudu/127.30.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.62:40601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:11.925045 4161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:11.925275 4161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:11.929638 4178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:11.929744 4173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:11.929639 4172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:11.929778 4161 server_base.cc:1034] running on GCE node
I20260808 17:06:11.930052 4161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:11.930346 4161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:11.931473 4161 hybrid_clock.cc:648] HybridClock initialized: now 1786208771931464 us; error 27 us; skew 500 ppm
Aug 08 17:06:11 dist-test-slave-2j7r krb5kdc[4134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208771, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:11.933674 4161 init.cc:377] Logged in from keytab as kudu/127.30.194.62@KRBTEST.COM (short username kudu)
I20260808 17:06:11.934067 4161 webserver.cc:460] Webserver started at http://127.30.194.62:33221/ using document root <none> and password file <none>
I20260808 17:06:11.934254 4161 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:11.934296 4161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:11.934413 4161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:11.935364 4161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/data/instance:
uuid: "03dcc1a457514133a11b379cd26b0bce"
format_stamp: "Formatted at 2026-08-08 17:06:11 on dist-test-slave-2j7r"
server_key: "5e68dd48306868b345c6987a2215a376"
server_key_iv: "7a15de0233c9b8be9274c82d82e75958"
server_key_version: "encryptionkey@0"
I20260808 17:06:11.935669 4161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/wal/instance:
uuid: "03dcc1a457514133a11b379cd26b0bce"
format_stamp: "Formatted at 2026-08-08 17:06:11 on dist-test-slave-2j7r"
server_key: "5e68dd48306868b345c6987a2215a376"
server_key_iv: "7a15de0233c9b8be9274c82d82e75958"
server_key_version: "encryptionkey@0"
I20260808 17:06:11.936822 4161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:06:11.937515 4192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:11.937659 4161 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:11.937740 4161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/wal
uuid: "03dcc1a457514133a11b379cd26b0bce"
format_stamp: "Formatted at 2026-08-08 17:06:11 on dist-test-slave-2j7r"
server_key: "5e68dd48306868b345c6987a2215a376"
server_key_iv: "7a15de0233c9b8be9274c82d82e75958"
server_key_version: "encryptionkey@0"
I20260808 17:06:11.937800 4161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:11.951112 4161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:11.952580 4161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:11.952699 4161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:11.956377 4161 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.62:40601
I20260808 17:06:11.956421 4281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.62:40601 every 8 connection(s)
I20260808 17:06:11.956692 4161 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/data/info.pb
I20260808 17:06:11.957598 4282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:11.958936 4282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:06:11 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208771, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:06:11.965919 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 4161
I20260808 17:06:11.966034 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-0/wal/instance
I20260808 17:06:11.966202 31496 external_mini_cluster.cc:1428] Setting key 7442f7621a4242996fecb250083f895c
I20260808 17:06:11.967267 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:11.959506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40601 (local address 127.0.0.1:47218)
0808 17:06:11.959698 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:11.959747 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:11.960190 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:11.960438 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:11.960441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:11.960687 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:11.960839 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:11.960843 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:11.961855 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:11.961856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:11.962488 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:11.962489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:11.962930 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:11.962932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:11.962991 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:11.963357 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:11.963365 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:11.964566 (+ 1201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:11.965919 (+ 1353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:11.965926 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:11.965928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:11.966160 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:11.966400 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:11.966401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:11.966402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:11.966483 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:11.966773 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:11.966778 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:11.966835 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 17:06:11.966914 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 17:06:11.967057 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260808 17:06:11.967273 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:11.959581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.30.194.62:40601)
0808 17:06:11.959707 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:11.959716 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:11.959833 (+ 117us) server_negotiation.cc:407] Connection header received
0808 17:06:11.960330 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:11.960332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:11.960348 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:11.960421 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:11.960926 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:11.961772 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:11.962561 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:11.962844 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:11.962935 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:11.964643 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:11.964650 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:11.964650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:11.964657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:11.965850 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:11.966256 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:11.966259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:11.966260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:11.966344 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:11.966533 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:11.966534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:11.966535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:11.966712 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:11.966781 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:11.966953 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 17:06:11.967121 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260808 17:06:11.967931 4282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:11.968216 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:11.968106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:46431 (local address 127.0.0.1:60874)
0808 17:06:11.968147 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:11.968193 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:46431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 17:06:11.968339 4197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.194.61:46431: connect: Connection refused (error 111)
I20260808 17:06:11.968683 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:11.968601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:46431 (local address 127.0.0.1:60886)
0808 17:06:11.968638 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:11.968664 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:46431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 17:06:11.968787 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.61:46431: Network error: Client connection negotiation failed: client connection to 127.30.194.61:46431: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.30.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab.
Entry for principal kudu/127.30.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab.
I20260808 17:06:11.989518 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.61:46431
--webserver_interface=127.30.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab
--principal=kudu/127.30.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:40601,127.30.194.61:46431,127.30.194.60:41603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:06:12.018007 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } attempt: 1
I20260808 17:06:12.018492 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.018360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:46431 (local address 127.0.0.1:60894)
0808 17:06:12.018427 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.018461 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:46431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 17:06:12.018832 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.018748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:46431 (local address 127.0.0.1:60904)
0808 17:06:12.018785 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.018812 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:46431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 17:06:12.018963 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.61:46431: Network error: Client connection negotiation failed: client connection to 127.30.194.61:46431: connect: Connection refused (error 111)
W20260808 17:06:12.068282 4294 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:06:12.068481 4294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:12.068498 4294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:12.068517 4294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:12.069650 4294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:12.069690 4294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:12.069703 4294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:12.069715 4294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:12.069729 4294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:12.071179 4294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:40601,127.30.194.61:46431,127.30.194.60:41603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.61.keytab
--principal=kudu/127.30.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.61:46431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:12.071408 4294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:12.071600 4294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:12.076284 4301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:12.076277 4308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:12.076514 4294 server_base.cc:1034] running on GCE node
W20260808 17:06:12.076287 4303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:12.076848 4294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:12.077174 4294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:12.078349 4294 hybrid_clock.cc:648] HybridClock initialized: now 1786208772078337 us; error 35 us; skew 500 ppm
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208772, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:12.080694 4294 init.cc:377] Logged in from keytab as kudu/127.30.194.61@KRBTEST.COM (short username kudu)
I20260808 17:06:12.081143 4294 webserver.cc:460] Webserver started at http://127.30.194.61:38631/ using document root <none> and password file <none>
I20260808 17:06:12.081336 4294 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:12.081374 4294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:12.081458 4294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:12.082376 4294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/data/instance:
uuid: "d9c67d39fed34b8d83661d92cb508a0b"
format_stamp: "Formatted at 2026-08-08 17:06:12 on dist-test-slave-2j7r"
server_key: "6e8a93fddb7cc2a67d59b62e813decd5"
server_key_iv: "ebc5a8ab180f93962424c2c080e050c5"
server_key_version: "encryptionkey@0"
I20260808 17:06:12.082659 4294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/wal/instance:
uuid: "d9c67d39fed34b8d83661d92cb508a0b"
format_stamp: "Formatted at 2026-08-08 17:06:12 on dist-test-slave-2j7r"
server_key: "6e8a93fddb7cc2a67d59b62e813decd5"
server_key_iv: "ebc5a8ab180f93962424c2c080e050c5"
server_key_version: "encryptionkey@0"
I20260808 17:06:12.083797 4294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:06:12.084412 4317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:12.084554 4294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:06:12.084636 4294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/wal
uuid: "d9c67d39fed34b8d83661d92cb508a0b"
format_stamp: "Formatted at 2026-08-08 17:06:12 on dist-test-slave-2j7r"
server_key: "6e8a93fddb7cc2a67d59b62e813decd5"
server_key_iv: "ebc5a8ab180f93962424c2c080e050c5"
server_key_version: "encryptionkey@0"
I20260808 17:06:12.084695 4294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:12.087100 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } attempt: 2
I20260808 17:06:12.087450 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.087350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:46431 (local address 127.0.0.1:60910)
0808 17:06:12.087399 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.087426 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:46431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 17:06:12.087770 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.087677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:46431 (local address 127.0.0.1:60924)
0808 17:06:12.087729 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.087751 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.61:46431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 17:06:12.087863 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.61:46431: Network error: Client connection negotiation failed: client connection to 127.30.194.61:46431: connect: Connection refused (error 111)
I20260808 17:06:12.093907 4294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:12.095788 4294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:12.095908 4294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:12.099519 4294 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.61:46431
I20260808 17:06:12.099560 4401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.61:46431 every 8 connection(s)
I20260808 17:06:12.099861 4294 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/data/info.pb
I20260808 17:06:12.100750 4402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:12.102160 4402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:12.103554 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 4294
I20260808 17:06:12.103665 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-1/wal/instance
I20260808 17:06:12.103826 31496 external_mini_cluster.cc:1428] Setting key 44a0b9d7f156e88c57739c04ab17c6ff
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208772, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:06:12.110999 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.102739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.30.194.62:40601)
0808 17:06:12.102813 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.102815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.103004 (+ 189us) server_negotiation.cc:407] Connection header received
0808 17:06:12.103448 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.103450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.103462 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.103508 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.104138 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.104605 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.105794 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.106010 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.106063 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.108361 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.108363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.108363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.108370 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.109704 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.110145 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.110145 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.110146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.110192 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.110452 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.110452 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.110452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.110547 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.110591 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.110883 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.110941 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 17:06:12.111225 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.102709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40601 (local address 127.0.0.1:47234)
0808 17:06:12.102894 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.102928 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:12.103348 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:12.103578 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:12.103582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:12.103849 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:12.104057 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.104062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.104684 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.104685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.105687 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.105690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.106098 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.106100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.106240 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.106747 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:12.106756 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:12.108254 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:12.109789 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.109797 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.109799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.110075 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.110242 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.110243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.110243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.110368 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.110602 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:12.110609 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:12.110762 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 17:06:12.110873 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 17:06:12.111019 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260808 17:06:12.111770 4402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.194.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.194.60@KRBTEST.COM" created.
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208772, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
I20260808 17:06:12.118148 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.111985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:46431 (local address 127.0.0.1:60926)
0808 17:06:12.112031 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.112042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:12.112082 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:12.112284 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:12.112285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:12.112423 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:12.112475 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.112476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.113285 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.113285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.113761 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.113763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.114142 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.114142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.114187 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.114478 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:12.114483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:12.115624 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:12.117184 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.117185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.117185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.117388 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.117559 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.117560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.117560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.117622 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.117837 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:12.117837 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:12.117880 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 17:06:12.117943 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 17:06:12.118034 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 17:06:12.118175 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.112022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.30.194.61:46431)
0808 17:06:12.112155 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.112159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.112174 (+ 15us) server_negotiation.cc:407] Connection header received
0808 17:06:12.112191 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.112192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.112202 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.112266 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.112541 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.113219 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.113816 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.114092 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.114137 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.115702 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.115710 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.115711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.115718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.117109 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.117456 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.117458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.117458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.117506 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.117675 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.117676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.117676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.117783 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.117835 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.117974 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.118087 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260808 17:06:12.118721 4402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:12.119001 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.118899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42840)
0808 17:06:12.118932 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.118978 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260808 17:06:12.119125 4323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
I20260808 17:06:12.119498 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.119400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42850)
0808 17:06:12.119445 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.119475 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 17:06:12.119594 4402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:41603: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab.
Entry for principal kudu/127.30.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab.
I20260808 17:06:12.128647 31496 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5D6OT5/build/release/bin/kudu
/tmp/dist-test-task5D6OT5/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.60:41603
--webserver_interface=127.30.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab
--principal=kudu/127.30.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.194.20:43247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.194.62:40601,127.30.194.61:46431,127.30.194.60:41603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:06:12.168797 4402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } attempt: 1
I20260808 17:06:12.169216 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.169076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42856)
0808 17:06:12.169143 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.169182 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 17:06:12.169523 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.169442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42868)
0808 17:06:12.169478 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.169503 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 17:06:12.169613 4402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:41603: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
I20260808 17:06:12.179005 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } attempt: 3
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208771, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
I20260808 17:06:12.185961 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.179255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:46431 (local address 127.0.0.1:60928)
0808 17:06:12.179307 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.179318 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:12.179441 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:12.179603 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:12.179604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:12.179739 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:12.179799 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.179801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.180681 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.180682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.181233 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.181236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.181702 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.181702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.181750 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.182147 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:12.182151 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:12.183435 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:12.185071 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.185072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.185072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.185275 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.185460 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.185460 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.185461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.185510 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.185729 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:12.185729 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:12.185764 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 17:06:12.185812 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 17:06:12.185886 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 17:06:12.186009 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.179434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.30.194.61:46431)
0808 17:06:12.179472 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.179474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.179484 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:06:12.179509 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.179509 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.179521 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.179583 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.179855 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.180617 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.181299 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.181637 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.181708 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.183522 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.183524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.183524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.183532 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.184984 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.185339 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.185340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.185341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.185401 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.185554 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.185555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.185555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.185678 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.185749 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.185837 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.185913 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:06:12.186458 4282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:12.186743 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.186660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42872)
0808 17:06:12.186697 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.186722 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 17:06:12.187032 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.186971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42876)
0808 17:06:12.186995 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.187017 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 17:06:12.187109 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:41603: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
W20260808 17:06:12.209061 4414 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:06:12.209312 4414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:06:12.209329 4414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:06:12.209347 4414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:06:12.210450 4414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:06:12.210485 4414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:06:12.210498 4414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:06:12.210518 4414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:06:12.210529 4414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:06:12.211925 4414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.194.20:43247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.194.62:40601,127.30.194.61:46431,127.30.194.60:41603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/krb5kdc/kudu_127.30.194.60.keytab
--principal=kudu/127.30.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.194.60:41603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:06:12.212114 4414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:06:12.212322 4414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:06:12.217001 4425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:12.217027 4414 server_base.cc:1034] running on GCE node
W20260808 17:06:12.216965 4423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:06:12.216980 4420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:06:12.217425 4414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:06:12.217669 4414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:06:12.218243 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } attempt: 1
I20260808 17:06:12.218612 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.218506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42888)
0808 17:06:12.218556 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.218587 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 17:06:12.218796 4414 hybrid_clock.cc:648] HybridClock initialized: now 1786208772218780 us; error 29 us; skew 500 ppm
I20260808 17:06:12.218888 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.218802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42900)
0808 17:06:12.218841 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.218868 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 17:06:12.218971 4282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:41603: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208772, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:06:12.221145 4414 init.cc:377] Logged in from keytab as kudu/127.30.194.60@KRBTEST.COM (short username kudu)
I20260808 17:06:12.221544 4414 webserver.cc:460] Webserver started at http://127.30.194.60:39167/ using document root <none> and password file <none>
I20260808 17:06:12.221710 4414 fs_manager.cc:359] Metadata directory not provided
I20260808 17:06:12.221746 4414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:06:12.221840 4414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:06:12.222810 4414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/data/instance:
uuid: "36116b3257694aacaca78ff7a5f89f08"
format_stamp: "Formatted at 2026-08-08 17:06:12 on dist-test-slave-2j7r"
server_key: "8a67f2ca07241237f3946629a99240e2"
server_key_iv: "b3327a0972ad6cbd781a3358f054b6e9"
server_key_version: "encryptionkey@0"
I20260808 17:06:12.223088 4414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/wal/instance:
uuid: "36116b3257694aacaca78ff7a5f89f08"
format_stamp: "Formatted at 2026-08-08 17:06:12 on dist-test-slave-2j7r"
server_key: "8a67f2ca07241237f3946629a99240e2"
server_key_iv: "b3327a0972ad6cbd781a3358f054b6e9"
server_key_version: "encryptionkey@0"
I20260808 17:06:12.224220 4414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:06:12.224838 4436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:06:12.224992 4414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:06:12.225073 4414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/data,/tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/wal
uuid: "36116b3257694aacaca78ff7a5f89f08"
format_stamp: "Formatted at 2026-08-08 17:06:12 on dist-test-slave-2j7r"
server_key: "8a67f2ca07241237f3946629a99240e2"
server_key_iv: "b3327a0972ad6cbd781a3358f054b6e9"
server_key_version: "encryptionkey@0"
I20260808 17:06:12.225159 4414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:06:12.237749 4402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } attempt: 2
I20260808 17:06:12.238108 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.237992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42904)
0808 17:06:12.238044 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.238080 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 17:06:12.238371 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.238302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42920)
0808 17:06:12.238333 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.238355 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 17:06:12.238471 4402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.194.60:41603: Network error: Client connection negotiation failed: client connection to 127.30.194.60:41603: connect: Connection refused (error 111)
I20260808 17:06:12.243616 4414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:06:12.245518 4414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:06:12.245620 4414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:06:12.249037 4414 rpc_server.cc:307] RPC server started. Bound to: 127.30.194.60:41603
I20260808 17:06:12.249068 4530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.194.60:41603 every 8 connection(s)
I20260808 17:06:12.249397 4414 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/data/info.pb
I20260808 17:06:12.250309 4531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:06:12.251695 4531 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:12.252804 31496 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5D6OT5/build/release/bin/kudu as pid 4414
I20260808 17:06:12.252925 31496 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5D6OT5/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208501350379-31496-0/minicluster-data/master-2/wal/instance
I20260808 17:06:12.253139 31496 external_mini_cluster.cc:1428] Setting key a04dd8e02d0e381dd9be4c0383b86ac8
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208772, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
I20260808 17:06:12.260218 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.252312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47240 (local address 127.30.194.62:40601)
0808 17:06:12.252368 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.252369 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.252541 (+ 172us) server_negotiation.cc:407] Connection header received
0808 17:06:12.253140 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.253142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.253154 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.253212 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.253908 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.254360 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.255267 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.255464 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.255512 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.257561 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.257562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.257563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.257570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.258987 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.259419 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.259421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.259422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.259485 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.259688 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.259689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.259689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.259819 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.259894 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.260093 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.260159 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 17:06:12.260447 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.252281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.62:40601 (local address 127.0.0.1:47240)
0808 17:06:12.252434 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.252467 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:12.253011 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:12.253279 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:12.253283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:12.253568 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:12.253838 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.253844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.254442 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.254443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.255188 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.255189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.255551 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.255552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.255677 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.256091 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:12.256100 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:12.257482 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:12.259067 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.259074 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.259075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.259352 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.259547 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.259547 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.259548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.259637 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.259879 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:12.259882 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:12.259980 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 17:06:12.260056 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 17:06:12.260189 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":19,"thread_start_us":72,"threads_started":1}
I20260808 17:06:12.260959 4531 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:12.261168 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.253520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.30.194.60:41603)
0808 17:06:12.253661 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.253665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.253700 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:06:12.253762 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.253763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.253779 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.253834 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.254433 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.255195 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.256064 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.256336 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.256387 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.258453 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.258461 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.258461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.258468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.260016 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.260366 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.260370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.260370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.260426 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.260617 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.260619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.260619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.260758 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.260885 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.260981 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.261048 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208772, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.62@KRBTEST.COM
I20260808 17:06:12.267438 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.261200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.61:46431 (local address 127.0.0.1:60938)
0808 17:06:12.261438 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.261445 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:12.261486 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:12.261617 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:12.261617 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:12.261731 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:12.261770 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.261771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.262609 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.262610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.263285 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.263286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.263719 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.263720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.263767 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.264070 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:12.264075 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:12.265119 (+ 1044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:12.266596 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.266597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.266597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.266784 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.266960 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.266961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.266961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.267019 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.267206 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:12.267207 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:12.267255 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 17:06:12.267314 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:06:12.267385 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228}
I20260808 17:06:12.267544 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.261274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60938 (local address 127.30.194.61:46431)
0808 17:06:12.261416 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.261418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.261492 (+ 74us) server_negotiation.cc:407] Connection header received
0808 17:06:12.261536 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.261537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.261547 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.261578 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.261868 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.262546 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.263349 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.263661 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.263723 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.265188 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.265189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.265190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.265197 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.266519 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.266849 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.266850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.266850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.266905 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.267073 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.267073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.267074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.267160 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.267200 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.267355 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.267422 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260808 17:06:12.267827 4531 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:06:12.268465 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.261836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47252 (local address 127.30.194.62:40601)
0808 17:06:12.261873 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.261874 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.261881 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:06:12.261898 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.261899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.261906 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.261944 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.262441 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.263072 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.263861 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.264071 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.264134 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.265983 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.265984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.265984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.265991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.267240 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.267601 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.267602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.267603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.267737 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.267941 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.267942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.267943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.268045 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.268257 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.268320 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.268386 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208772, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.60@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.194.61@KRBTEST.COM
I20260808 17:06:12.273751 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.268043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42940)
0808 17:06:12.268081 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.268091 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:12.268124 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:12.268221 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:12.268221 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:12.268337 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:12.268359 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.268360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.268978 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.268979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.269670 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.269673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.270038 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.270039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.270076 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.270376 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:12.270380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:12.271539 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:12.272824 (+ 1285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.272824 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.272825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.272994 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.273206 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.273207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.273207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.273287 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.273497 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:12.273497 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:12.273538 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 17:06:12.273608 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 17:06:12.273689 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 17:06:12.273842 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.268109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42940 (local address 127.30.194.60:41603)
0808 17:06:12.268140 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.268142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.268148 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:06:12.268163 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.268163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.268171 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.268219 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.268416 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.268914 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.269752 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.269973 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.270034 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.271603 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.271604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.271605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.271610 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.272745 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.273057 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.273058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.273058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.273143 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.273339 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.273340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.273340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.273437 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.273485 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.273624 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.273746 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":16}
I20260808 17:06:12.275312 4531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08: Bootstrap starting.
I20260808 17:06:12.276067 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.269274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.30.194.61:46431)
0808 17:06:12.269305 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.269306 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.269333 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:06:12.269350 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.269351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.269359 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.269391 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.269872 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.270384 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.271124 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.271438 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.271503 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.273676 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.273677 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.273678 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.273683 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.275101 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.275395 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.275396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.275396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.275451 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.275611 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.275611 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.275612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.275731 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.275887 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.275924 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.275992 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 17:06:12.276237 4531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:12.276501 4531 log.cc:824] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:12.277307 4531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08: No bootstrap required, opened a new log
I20260808 17:06:12.277355 31496 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:06:12.278108 4236 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "03dcc1a457514133a11b379cd26b0bce"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47252
I20260808 17:06:12.278705 4531 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } }
I20260808 17:06:12.278824 4531 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36116b3257694aacaca78ff7a5f89f08, State: Initialized, Role: FOLLOWER
I20260808 17:06:12.278920 4531 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } }
I20260808 17:06:12.279196 4531 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:12.279183 4545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } } }
I20260808 17:06:12.279296 4545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 17:06:12.281025 4564 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:06:12.281080 4564 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:06:12.281411 4414 master_runner.cc:186] Error getting master registration for 127.30.194.62:40601: OK, instance_id { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" instance_seqno: 1786208771955384 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:06:12.283314 4414 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:06:12.294092 4282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } attempt: 2
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208771, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.62@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
I20260808 17:06:12.300148 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.294421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42942 (local address 127.30.194.60:41603)
0808 17:06:12.294460 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.294462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.294470 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:06:12.294493 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.294493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.294503 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.294562 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.294827 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.295285 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.295957 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.296154 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.296219 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.297892 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.297894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.297894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.297900 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.299107 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.299433 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.299434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.299435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.299486 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.299659 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.299660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.299660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.299781 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.299844 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.299989 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.300056 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:06:12.300153 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.294349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42942)
0808 17:06:12.294395 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.294404 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:12.294444 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:12.294572 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:12.294572 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:12.294708 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:12.294745 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.294746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.295364 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.295365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.295873 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.295875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.296233 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.296234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.296306 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.296692 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:12.296697 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:12.297801 (+ 1104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:12.299190 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.299191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.299192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.299359 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.299542 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.299543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.299543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.299604 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.299845 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:12.299846 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:12.299892 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 17:06:12.299960 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 17:06:12.300056 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 17:06:12.301434 4282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce: Bootstrap starting.
I20260808 17:06:12.301996 4282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:12.302233 4282 log.cc:824] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:12.302937 4282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce: No bootstrap required, opened a new log
I20260808 17:06:12.303891 4282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } }
I20260808 17:06:12.303997 4282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03dcc1a457514133a11b379cd26b0bce, State: Initialized, Role: FOLLOWER
I20260808 17:06:12.304056 4282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } }
I20260808 17:06:12.304333 4282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:12.304304 4570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } } }
I20260808 17:06:12.304383 4570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:12.304464 4236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 17:06:12.304569 4236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:06:12.305365 4236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } }
I20260808 17:06:12.306023 4491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03dcc1a457514133a11b379cd26b0bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "36116b3257694aacaca78ff7a5f89f08"
I20260808 17:06:12.306140 4491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [term 0 FOLLOWER]: Advancing to term 1
W20260808 17:06:12.306489 4593 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:06:12.306538 4593 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:06:12.306970 4491 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03dcc1a457514133a11b379cd26b0bce in term 1.
I20260808 17:06:12.307581 4197 leader_election.cc:302] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03dcc1a457514133a11b379cd26b0bce, 36116b3257694aacaca78ff7a5f89f08; no voters:
I20260808 17:06:12.307714 4570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:06:12.307881 4570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [term 1 LEADER]: Becoming Leader. State: Replica: 03dcc1a457514133a11b379cd26b0bce, State: Running, Role: LEADER
I20260808 17:06:12.307963 4570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } }
I20260808 17:06:12.308373 4570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03dcc1a457514133a11b379cd26b0bce. Latest consensus state: current_term: 1 leader_uuid: "03dcc1a457514133a11b379cd26b0bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } } }
I20260808 17:06:12.308457 4570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:12.308768 4598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:06:12.309136 4598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:06:12.307744 4161 master_runner.cc:186] Error getting master registration for 127.30.194.61:46431: OK, instance_id { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" instance_seqno: 1786208772098508 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:06:12.309924 4161 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:06:12.310101 4236 leader_election.cc:288] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [CANDIDATE]: Term 1 election: Requested vote from peers d9c67d39fed34b8d83661d92cb508a0b (127.30.194.61:46431), 36116b3257694aacaca78ff7a5f89f08 (127.30.194.60:41603)
I20260808 17:06:12.313409 4491 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [term 1 FOLLOWER]: Refusing update from remote peer 03dcc1a457514133a11b379cd26b0bce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:06:12.313769 4595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:06:12.315214 4545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03dcc1a457514133a11b379cd26b0bce. Latest consensus state: current_term: 1 leader_uuid: "03dcc1a457514133a11b379cd26b0bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } } }
I20260808 17:06:12.315292 4545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:12.322151 4595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03dcc1a457514133a11b379cd26b0bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } } }
I20260808 17:06:12.322243 4595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:12.322728 4545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03dcc1a457514133a11b379cd26b0bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } } }
I20260808 17:06:12.322800 4545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:12.323052 4603 mvcc.cc:205] Tried to move back new op lower bound from 7316311131391778816 to 7316311131374174208. Current Snapshot: MvccSnapshot[applied={T|T < 7316311131391778816}]
I20260808 17:06:12.324190 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.311119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.30.194.62:40601)
0808 17:06:12.311154 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.311155 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.311401 (+ 246us) server_negotiation.cc:407] Connection header received
0808 17:06:12.311492 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.311493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.311502 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.311529 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.312275 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.312978 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.316178 (+ 3200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.316479 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.316515 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.319023 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.319024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.319024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.319032 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.321026 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.323037 (+ 2011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.323038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.323039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.323102 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.323501 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.323502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.323502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.323641 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.323732 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.324030 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.324105 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:06:12.326336 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.311826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42948 (local address 127.30.194.60:41603)
0808 17:06:12.311863 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.311865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.316878 (+ 5013us) server_negotiation.cc:407] Connection header received
0808 17:06:12.316966 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.316967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.316976 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.317002 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.317703 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.318387 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.319352 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.319643 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.319678 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.320961 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.320962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.320963 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.320969 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.322590 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.325276 (+ 2686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.325277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.325277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.325351 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.325661 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.325662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.325663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.325794 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.325876 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.326180 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.326251 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:06:12.326833 4598 catalog_manager.cc:1347] Generated new cluster ID: f0ef8bb3b8cf404fa10a705d9c0abad6
I20260808 17:06:12.326874 4598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:06:12.330109 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.311359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.30.194.61:46431)
0808 17:06:12.311402 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.311404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.313570 (+ 2166us) server_negotiation.cc:407] Connection header received
0808 17:06:12.313752 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.313767 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.313779 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.313811 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.315223 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.315871 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.324022 (+ 8151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.324324 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.324369 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.326577 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.326591 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.326592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.326599 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.328371 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.328885 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.328886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.328887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.328953 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.329211 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.329212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.329212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.329351 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.329909 (+ 558us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.329949 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.330024 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:06:12.330174 4402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } attempt: 3
Aug 08 17:06:12 dist-test-slave-2j7r krb5kdc[4134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208772, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.194.61@KRBTEST.COM for kudu/127.30.194.60@KRBTEST.COM
I20260808 17:06:12.338039 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.330421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.194.60:41603 (local address 127.0.0.1:42958)
0808 17:06:12.330604 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 17:06:12.330614 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:06:12.330652 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:06:12.330950 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:06:12.330951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:06:12.331085 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:06:12.331119 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.331120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.331977 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.331978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.332736 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:06:12.332739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.333506 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.333507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:06:12.333569 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.333911 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:06:12.333915 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:06:12.335280 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:06:12.336977 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.336977 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.336978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.337224 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.337454 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:06:12.337455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:06:12.337456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:06:12.337521 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:06:12.337780 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:06:12.337781 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:06:12.337819 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 17:06:12.337871 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 17:06:12.337951 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260808 17:06:12.338065 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:12.330738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.30.194.60:41603)
0808 17:06:12.330840 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.330842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.330849 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:06:12.330866 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.330867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.330876 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.330935 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.331182 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.331840 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.332817 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:12.333113 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:12.333168 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:12.335370 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:06:12.335372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:06:12.335372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:06:12.335398 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:06:12.336895 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.337318 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.337320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.337320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.337381 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:06:12.337582 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:06:12.337583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:06:12.337583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:06:12.337715 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:06:12.337780 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:12.337905 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 17:06:12.337977 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 17:06:12.339653 4402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b: Bootstrap starting.
I20260808 17:06:12.340287 4402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b: Neither blocks nor log segments found. Creating new log.
I20260808 17:06:12.340569 4402 log.cc:824] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b: Log is configured to *not* fsync() on all Append() calls
I20260808 17:06:12.341434 4402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b: No bootstrap required, opened a new log
I20260808 17:06:12.343322 4402 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } }
I20260808 17:06:12.343449 4402 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c67d39fed34b8d83661d92cb508a0b, State: Initialized, Role: FOLLOWER
I20260808 17:06:12.343511 4402 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } }
I20260808 17:06:12.343796 4402 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:06:12.344391 4620 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } } }
I20260808 17:06:12.344480 4620 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:12.345225 4363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03dcc1a457514133a11b379cd26b0bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d9c67d39fed34b8d83661d92cb508a0b"
I20260808 17:06:12.345332 4363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [term 0 FOLLOWER]: Advancing to term 1
W20260808 17:06:12.345547 4647 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:06:12.345595 4647 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:06:12.346524 4363 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03dcc1a457514133a11b379cd26b0bce in term 1.
W20260808 17:06:12.346671 4329 tablet.cc:2369] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:06:12.346994 4362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [term 1 FOLLOWER]: Refusing update from remote peer 03dcc1a457514133a11b379cd26b0bce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:06:12.347393 4570 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:06:12.348613 4620 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03dcc1a457514133a11b379cd26b0bce. Latest consensus state: current_term: 1 leader_uuid: "03dcc1a457514133a11b379cd26b0bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } } }
I20260808 17:06:12.348717 4620 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:12.348757 4620 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03dcc1a457514133a11b379cd26b0bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } } }
I20260808 17:06:12.348809 4620 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:06:12.358680 4650 mvcc.cc:205] Tried to move back new op lower bound from 7316311131391778816 to 7316311131374174208. Current Snapshot: MvccSnapshot[applied={T|T < 7316311131391778816}]
I20260808 17:06:12.361910 4598 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:06:12.362581 4598 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:06:12.365870 4595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03dcc1a457514133a11b379cd26b0bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03dcc1a457514133a11b379cd26b0bce" member_type: VOTER last_known_addr { host: "127.30.194.62" port: 40601 } } peers { permanent_uuid: "d9c67d39fed34b8d83661d92cb508a0b" member_type: VOTER last_known_addr { host: "127.30.194.61" port: 46431 } } peers { permanent_uuid: "36116b3257694aacaca78ff7a5f89f08" member_type: VOTER last_known_addr { host: "127.30.194.60" port: 41603 } } }
I20260808 17:06:12.366003 4595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce [sys.catalog]: This master's current role is: LEADER
I20260808 17:06:12.384424 4598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03dcc1a457514133a11b379cd26b0bce: Generated new TSK 0
I20260808 17:06:12.384610 4598 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:06:12.414541 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.412858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.30.194.61:46431)
0808 17:06:12.412915 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.412917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.413862 (+ 945us) server_negotiation.cc:407] Connection header received
0808 17:06:12.413987 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.413988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.414000 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.414028 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.414474 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60966: BlockingRecv error: recv got EOF from 127.0.0.1:60966 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 17:06:12.415309 4286 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.413136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47276 (local address 127.30.194.62:40601)
0808 17:06:12.413177 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.413179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.414758 (+ 1579us) server_negotiation.cc:407] Connection header received
0808 17:06:12.414850 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.414851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.414862 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.414901 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.415249 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47276: BlockingRecv error: recv got EOF from 127.0.0.1:47276 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 17:06:12.416121 4539 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.413495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42974 (local address 127.30.194.60:41603)
0808 17:06:12.413535 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.413536 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.415551 (+ 2015us) server_negotiation.cc:407] Connection header received
0808 17:06:12.415708 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.415709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.415720 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.415750 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.416065 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42974: BlockingRecv error: recv got EOF from 127.0.0.1:42974 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 17:06:12.418355 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.417589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60976 (local address 127.30.194.61:46431)
0808 17:06:12.417633 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.417635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.417841 (+ 206us) server_negotiation.cc:407] Connection header received
0808 17:06:12.417961 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.417962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.417980 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.418010 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.418314 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60976: BlockingRecv error: recv got EOF from 127.0.0.1:60976 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 17:06:12.421245 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.420476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.30.194.61:46431)
0808 17:06:12.420506 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.420508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.420741 (+ 233us) server_negotiation.cc:407] Connection header received
0808 17:06:12.420832 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.420833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.420843 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.420870 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.421209 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60990: BlockingRecv error: recv got EOF from 127.0.0.1:60990 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 17:06:12.426240 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.425499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60996 (local address 127.30.194.61:46431)
0808 17:06:12.425538 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.425540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.425763 (+ 223us) server_negotiation.cc:407] Connection header received
0808 17:06:12.425859 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.425860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.425869 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.425905 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.426204 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60996: BlockingRecv error: recv got EOF from 127.0.0.1:60996 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 17:06:12.431900 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.431176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.30.194.61:46431)
0808 17:06:12.431205 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.431207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.431424 (+ 217us) server_negotiation.cc:407] Connection header received
0808 17:06:12.431511 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.431511 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.431520 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.431548 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.431841 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32772: BlockingRecv error: recv got EOF from 127.0.0.1:32772 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 17:06:12.441951 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.441006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32776 (local address 127.30.194.61:46431)
0808 17:06:12.441043 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.441044 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.441380 (+ 336us) server_negotiation.cc:407] Connection header received
0808 17:06:12.441486 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.441487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.441499 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.441530 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.441907 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32776: BlockingRecv error: recv got EOF from 127.0.0.1:32776 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 17:06:12.459697 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.458988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32792 (local address 127.30.194.61:46431)
0808 17:06:12.459021 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.459022 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.459242 (+ 220us) server_negotiation.cc:407] Connection header received
0808 17:06:12.459325 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.459326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.459336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.459361 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.459650 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32792: BlockingRecv error: recv got EOF from 127.0.0.1:32792 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 17:06:12.493507 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.492828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32794 (local address 127.30.194.61:46431)
0808 17:06:12.492881 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.492883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.492984 (+ 101us) server_negotiation.cc:407] Connection header received
0808 17:06:12.493037 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.493038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.493049 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.493112 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.493452 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32794: BlockingRecv error: recv got EOF from 127.0.0.1:32794 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 17:06:12.559347 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.558692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.30.194.61:46431)
0808 17:06:12.558759 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.558761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.558867 (+ 106us) server_negotiation.cc:407] Connection header received
0808 17:06:12.558921 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.558923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.558933 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.558982 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.559283 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32802: BlockingRecv error: recv got EOF from 127.0.0.1:32802 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 17:06:12.689237 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.688524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.30.194.61:46431)
0808 17:06:12.688600 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.688602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.688678 (+ 76us) server_negotiation.cc:407] Connection header received
0808 17:06:12.688759 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.688760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.688771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.688821 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.689175 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32810: BlockingRecv error: recv got EOF from 127.0.0.1:32810 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 17:06:12.946915 4410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:06:12.946360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.30.194.61:46431)
0808 17:06:12.946419 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:12.946421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:12.946431 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:06:12.946474 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:12.946474 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:12.946484 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:12.946536 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:06:12.946854 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32820: BlockingRecv error: recv got EOF from 127.0.0.1:32820 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:06:13.281510 4564 catalog_manager.cc:1259] Loaded cluster ID: f0ef8bb3b8cf404fa10a705d9c0abad6
I20260808 17:06:13.281570 4564 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08: loading cluster ID for follower catalog manager: success
I20260808 17:06:13.282308 4564 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08: acquiring CA information for follower catalog manager: success
I20260808 17:06:13.282550 4564 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36116b3257694aacaca78ff7a5f89f08: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:06:13.346035 4647 catalog_manager.cc:1259] Loaded cluster ID: f0ef8bb3b8cf404fa10a705d9c0abad6
I20260808 17:06:13.346107 4647 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b: loading cluster ID for follower catalog manager: success
I20260808 17:06:13.346884 4647 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b: acquiring CA information for follower catalog manager: success
I20260808 17:06:13.347152 4647 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d9c67d39fed34b8d83661d92cb508a0b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:06:13.462260 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:13.460020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.30.194.61:46431)
0808 17:06:13.460127 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:13.460128 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:13.460138 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:06:13.460189 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:13.460190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:13.460241 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:13.460312 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:06:13.460433 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:13.460929 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:13.461639 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:13.461836 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:13.461897 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:13.461990 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:06:13.462100 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:06:13.462152 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:13.462164 (+ 12us) server_negotiation.cc:299] Negotiation successful
0808 17:06:13.462181 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
I20260808 17:06:13.465953 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:13.463868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42986 (local address 127.30.194.60:41603)
0808 17:06:13.464004 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:13.464005 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:13.464013 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:06:13.464047 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:13.464047 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:13.464080 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:13.464141 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:06:13.464238 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:13.464691 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:13.465381 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:13.465563 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:13.465610 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:13.465700 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:06:13.465789 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:06:13.465837 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:13.465852 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 17:06:13.465866 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260808 17:06:13.470000 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:13.467581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32840 (local address 127.30.194.61:46431)
0808 17:06:13.467617 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:13.467618 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:13.467831 (+ 213us) server_negotiation.cc:407] Connection header received
0808 17:06:13.467856 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:13.467857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:13.467868 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:13.467916 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:06:13.468044 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:13.468484 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:13.469311 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:13.469579 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:13.469663 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:13.469768 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:06:13.469800 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:06:13.469857 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:13.469906 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 17:06:13.469928 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:06:13.470531 4777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:13.467470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47286 (local address 127.30.194.62:40601)
0808 17:06:13.467727 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:13.467729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:13.467739 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:06:13.467786 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:13.467787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:13.467838 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:13.467926 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:06:13.468134 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:13.468850 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:13.469674 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:13.469983 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:13.470039 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:13.470253 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:06:13.470331 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:06:13.470388 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:13.470450 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 17:06:13.470472 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":68,"threads_started":1}
I20260808 17:06:13.471968 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 4161
I20260808 17:06:13.478871 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:06:13.467678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43000 (local address 127.30.194.60:41603)
0808 17:06:13.476274 (+ 8596us) server_negotiation.cc:206] Beginning negotiation
0808 17:06:13.476276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:06:13.476286 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:06:13.476307 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:06:13.476307 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:06:13.476316 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:06:13.476370 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:06:13.476531 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:13.476988 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:13.478025 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:06:13.478405 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:06:13.478503 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:06:13.478624 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:06:13.478661 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:06:13.478724 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:06:13.478777 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 17:06:13.478799 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8565}
I20260808 17:06:13.486652 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 4294
I20260808 17:06:13.497143 31496 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5D6OT5/build/release/bin/kudu with pid 4414
2026-08-08T17:06:13Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2919 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5791 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (272159 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST